[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:56:50 dist-test-slave-jpj1 krb5kdc[22929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:56:50 dist-test-slave-jpj1 krb5kdc[22929](info): set up 2 sockets
Aug 06 08:56:50 dist-test-slave-jpj1 krb5kdc[22929](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 08:56:51.782459 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:56:51 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:56:51.980844 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42515
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:52.421432 22949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:52.421948 22949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:52.422343 22949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:52.430794 22949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:56:52.430960 22949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:52.431061 22949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:52.431144 22949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:56:52.431208 22949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:56:52.442625 22949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:52.444780 22949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:52.446864 22949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:52.473958 22959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:52.474538 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:52.474598 22958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:52.477162 22949 server_base.cc:1034] running on GCE node
I20260806 08:56:52.478543 22949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:52.481170 22949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:52.482528 22949 hybrid_clock.cc:648] HybridClock initialized: now 1786006612482438 us; error 113 us; skew 500 ppm
I20260806 08:56:52.483175 22949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:52 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:52.490583 22949 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:56:52.493525 22949 webserver.cc:460] Webserver started at http://127.22.98.62:40189/ using document root <none> and password file <none>
I20260806 08:56:52.494710 22949 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:52.494849 22949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:52.495396 22949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:52.500254 22949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "3b8aefa1f961488fa045257f3fb43c9a"
format_stamp: "Formatted at 2026-08-06 08:56:52 on dist-test-slave-jpj1"
server_key: "fff950996b0fb01857f62986a5ee750b"
server_key_iv: "dd6fbff3073bbf4594a068109816dfeb"
server_key_version: "encryptionkey@0"
I20260806 08:56:52.501195 22949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "3b8aefa1f961488fa045257f3fb43c9a"
format_stamp: "Formatted at 2026-08-06 08:56:52 on dist-test-slave-jpj1"
server_key: "fff950996b0fb01857f62986a5ee750b"
server_key_iv: "dd6fbff3073bbf4594a068109816dfeb"
server_key_version: "encryptionkey@0"
I20260806 08:56:52.510013 22949 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260806 08:56:52.516448 22972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:52.518796 22949 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.002s
I20260806 08:56:52.519117 22949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "3b8aefa1f961488fa045257f3fb43c9a"
format_stamp: "Formatted at 2026-08-06 08:56:52 on dist-test-slave-jpj1"
server_key: "fff950996b0fb01857f62986a5ee750b"
server_key_iv: "dd6fbff3073bbf4594a068109816dfeb"
server_key_version: "encryptionkey@0"
I20260806 08:56:52.519399 22949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:52.546411 22949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:52.552572 22949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:52.552968 22949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:52.580832 22949 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:42515
I20260806 08:56:52.580871 23041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:42515 every 8 connection(s)
I20260806 08:56:52.583425 22949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:56:52.586393 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22949
I20260806 08:56:52.586849 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:56:52.590085 22920 external_mini_cluster.cc:1428] Setting key d5d37ab341259a327ddc03ac8fc45f21
I20260806 08:56:52.593098 23043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:56:52.616328 23043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a: Bootstrap starting.
I20260806 08:56:52.624725 23043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a: Neither blocks nor log segments found. Creating new log.
I20260806 08:56:52.627342 23043 log.cc:824] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a: Log is configured to *not* fsync() on all Append() calls
I20260806 08:56:52.631985 23043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a: No bootstrap required, opened a new log
Aug 06 08:56:52 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:52.645627 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:52.598284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.22.98.62:42515)
0806 08:56:52.602022 (+ 3738us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:52.602036 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:52.602090 (+ 54us) server_negotiation.cc:407] Connection header received
0806 08:56:52.614881 (+ 12791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:52.614946 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:52.615641 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:52.616109 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:52.621175 (+ 5066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:52.622793 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:52.625859 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:52.626712 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:52.626944 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:52.633941 (+ 6997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:52.634026 (+ 85us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:52.634078 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:52.634149 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:52.638827 (+ 4678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:52.639863 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:52.639874 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:52.639883 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:52.640160 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:52.640626 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:52.640634 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:52.640638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:52.641687 (+ 1049us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:52.642124 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:52.643161 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0806 08:56:52.643655 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3373,"thread_start_us":2923,"threads_started":1}
I20260806 08:56:52.645150 23043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } }
I20260806 08:56:52.645885 23043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:56:52.646101 23043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8aefa1f961488fa045257f3fb43c9a, State: Initialized, Role: FOLLOWER
I20260806 08:56:52.647716 23043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } }
I20260806 08:56:52.648140 23043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:56:52.648449 23043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:56:52.648725 23043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:56:52.651950 23043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } }
I20260806 08:56:52.653072 23043 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b8aefa1f961488fa045257f3fb43c9a; no voters:
I20260806 08:56:52.653955 23043 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:56:52.654183 23050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:56:52.655071 23050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [term 1 LEADER]: Becoming Leader. State: Replica: 3b8aefa1f961488fa045257f3fb43c9a, State: Running, Role: LEADER
I20260806 08:56:52.655987 23050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } }
I20260806 08:56:52.657231 23043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:56:52.662460 23052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b8aefa1f961488fa045257f3fb43c9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } } }
I20260806 08:56:52.663045 23052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [sys.catalog]: This master's current role is: LEADER
I20260806 08:56:52.662905 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b8aefa1f961488fa045257f3fb43c9a. Latest consensus state: current_term: 1 leader_uuid: "3b8aefa1f961488fa045257f3fb43c9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8aefa1f961488fa045257f3fb43c9a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42515 } } }
I20260806 08:56:52.663403 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a [sys.catalog]: This master's current role is: LEADER
I20260806 08:56:52.664656 23064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:56:52.674582 23064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:56:52.695423 23064 catalog_manager.cc:1347] Generated new cluster ID: 99a70560dfee45888badc862ae190c30
I20260806 08:56:52.695667 23064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:56:52.711843 23064 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:56:52.713711 23064 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:56:52.725666 23064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b8aefa1f961488fa045257f3fb43c9a: Generated new TSK 0
I20260806 08:56:52.727082 23064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:56:52.787300 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:53.183800 23075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:53.184142 23075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:53.184283 23075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:53.192297 23075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:53.192438 23075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:53.192597 23075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:56:53.203024 23075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:53.204527 23075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:53.205929 23075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:53.220817 23089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:53.220904 23086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:53.220880 23082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:53.221957 23075 server_base.cc:1034] running on GCE node
I20260806 08:56:53.222638 23075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:53.223717 23075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:53.224953 23075 hybrid_clock.cc:648] HybridClock initialized: now 1786006613224891 us; error 52 us; skew 500 ppm
I20260806 08:56:53.225311 23075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:53 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:53.229024 23075 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:56:53.230872 23075 webserver.cc:460] Webserver started at http://127.22.98.1:35327/ using document root <none> and password file <none>
I20260806 08:56:53.231714 23075 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:53.231851 23075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:53.232180 23075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:53.235618 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "99ca22fc93dc494e904926a0182c4b2e"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "62c31667568f8691d2a6e71bc30c20b0"
server_key_iv: "0e6d1f806506d851e14d758498400fad"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.236536 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "99ca22fc93dc494e904926a0182c4b2e"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "62c31667568f8691d2a6e71bc30c20b0"
server_key_iv: "0e6d1f806506d851e14d758498400fad"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.242759 23075 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 08:56:53.247274 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.248996 23075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 08:56:53.249178 23075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "99ca22fc93dc494e904926a0182c4b2e"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "62c31667568f8691d2a6e71bc30c20b0"
server_key_iv: "0e6d1f806506d851e14d758498400fad"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.249398 23075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:53.272879 23075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:53.277647 23075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:53.277961 23075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:53.279433 23075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:53.281522 23075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:53.281646 23075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.281836 23075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:53.281903 23075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.325732 23075 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:36721
I20260806 08:56:53.325739 23270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:36721 every 8 connection(s)
I20260806 08:56:53.327152 23075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:56:53.330226 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 23075
I20260806 08:56:53.330492 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:56:53.331327 22920 external_mini_cluster.cc:1428] Setting key 48e93c4d7ca5acbbf88ccd31e9260a9a
Aug 06 08:56:53 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
I20260806 08:56:53.352434 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:53.330841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:34433 (local address 127.22.98.62:42515)
0806 08:56:53.331499 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:53.331506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:53.332732 (+ 1226us) server_negotiation.cc:407] Connection header received
0806 08:56:53.334815 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:53.334822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:53.334915 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:53.335032 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:53.337236 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.338187 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.340069 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.340677 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.340789 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:53.345535 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:53.345541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:53.345545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:53.345583 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:53.348751 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:53.349782 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:53.349787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:53.349790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:53.349942 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:53.350405 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:53.350409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:53.350411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:53.350717 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:53.350861 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:53.351742 (+ 881us) server_negotiation.cc:299] Negotiation successful
0806 08:56:53.351992 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":367,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
I20260806 08:56:53.353763 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:53.331452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42515 (local address 127.22.98.1:34433)
0806 08:56:53.332554 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:53.332613 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:53.334445 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:53.335358 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:53.335369 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:53.336066 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:53.337030 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:53.337057 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.338408 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.338412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:53.339837 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:53.339855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.341049 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.341059 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:53.341485 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:53.342255 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:53.342287 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:53.345213 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:53.348955 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:53.348968 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:53.348983 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:53.349603 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:53.350059 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:53.350064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:53.350067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:53.350279 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:53.350851 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:53.350860 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:53.351261 (+ 401us) client_negotiation.cc:769] Sending connection context
0806 08:56:53.351624 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 08:56:53.352260 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":523,"threads_started":1}
I20260806 08:56:53.355921 23273 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42515
I20260806 08:56:53.356431 23273 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:53.357452 23273 heartbeater.cc:511] Master 127.22.98.62:42515 requested a full tablet report, sending...
I20260806 08:56:53.361747 22995 ts_manager.cc:194] Registered new tserver with Master: 99ca22fc93dc494e904926a0182c4b2e (127.22.98.1:36721)
I20260806 08:56:53.364145 22995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:34433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:56:53.378329 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:53.773310 23285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:53.773610 23285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:53.773747 23285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:53.781348 23285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:53.781466 23285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:53.781623 23285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:56:53.792583 23285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:53.794003 23285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:53.795435 23285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:53.809504 23293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:53.809540 23296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:53.809688 23299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:53.813050 23285 server_base.cc:1034] running on GCE node
I20260806 08:56:53.813912 23285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:53.814987 23285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:53.816192 23285 hybrid_clock.cc:648] HybridClock initialized: now 1786006613816142 us; error 49 us; skew 500 ppm
I20260806 08:56:53.816514 23285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:53 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:53.819815 23285 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:56:53.821657 23285 webserver.cc:460] Webserver started at http://127.22.98.2:36203/ using document root <none> and password file <none>
I20260806 08:56:53.822376 23285 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:53.822497 23285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:53.822814 23285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:53.826007 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "c4a66cbdbda54632a761f87261179851"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "fd278163a8bad2be83fda532dfb4722f"
server_key_iv: "7e55db684f035bebdd18ac35bff08f5c"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.826851 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "c4a66cbdbda54632a761f87261179851"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "fd278163a8bad2be83fda532dfb4722f"
server_key_iv: "7e55db684f035bebdd18ac35bff08f5c"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.832304 23285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 08:56:53.836117 23309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.837474 23285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 08:56:53.837630 23285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "c4a66cbdbda54632a761f87261179851"
format_stamp: "Formatted at 2026-08-06 08:56:53 on dist-test-slave-jpj1"
server_key: "fd278163a8bad2be83fda532dfb4722f"
server_key_iv: "7e55db684f035bebdd18ac35bff08f5c"
server_key_version: "encryptionkey@0"
I20260806 08:56:53.837847 23285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:53.861465 23285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:53.866295 23285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:53.866575 23285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:53.867710 23285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:53.869323 23285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:53.869414 23285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.869544 23285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:53.869611 23285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:53.909618 23285 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:43091
I20260806 08:56:53.909662 23475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:43091 every 8 connection(s)
I20260806 08:56:53.911056 23285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:56:53.919514 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 23285
I20260806 08:56:53.919778 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:56:53.920282 22920 external_mini_cluster.cc:1428] Setting key d70dab498290f894a9d78f18f59e5805
Aug 06 08:56:53 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:53.932634 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:53.913901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:55561 (local address 127.22.98.62:42515)
0806 08:56:53.914587 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:53.914594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:53.915672 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 08:56:53.917660 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:53.917665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:53.917755 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:53.917856 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:53.919657 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.920494 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.922565 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.923145 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.923272 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:53.926961 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:53.926967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:53.926970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:53.927003 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:53.929504 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:53.930226 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:53.930230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:53.930232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:53.930350 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:53.930772 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:53.930775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:53.930778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:53.931043 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:53.931187 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:53.932025 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 08:56:53.932264 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":406,"threads_started":1}
I20260806 08:56:53.933812 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:53.914576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42515 (local address 127.22.98.2:55561)
0806 08:56:53.915493 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:53.915560 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:53.917378 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:53.918088 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:53.918097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:53.918630 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:53.919455 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:53.919478 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.920640 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.920644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:53.922430 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:53.922443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:53.923388 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:53.923395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:53.923660 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:53.924360 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:53.924388 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:53.926725 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:53.929646 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:53.929656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:53.929682 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:53.930099 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:53.930451 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:53.930456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:53.930460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:53.930650 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:53.931169 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:53.931177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:53.931601 (+ 424us) client_negotiation.cc:769] Sending connection context
0806 08:56:53.931907 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 08:56:53.932568 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":463,"threads_started":1}
I20260806 08:56:53.935245 23476 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42515
I20260806 08:56:53.935635 23476 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:53.936553 23476 heartbeater.cc:511] Master 127.22.98.62:42515 requested a full tablet report, sending...
I20260806 08:56:53.938470 22995 ts_manager.cc:194] Registered new tserver with Master: c4a66cbdbda54632a761f87261179851 (127.22.98.2:43091)
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
I20260806 08:56:53.939405 22995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:55561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:56:53.964097 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:54.354190 23485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:54.354583 23485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:54.354839 23485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:54.363013 23485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:54.363137 23485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:54.363333 23485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:56:54.368224 23273 heartbeater.cc:503] Master 127.22.98.62:42515 was elected leader, sending a full tablet report...
I20260806 08:56:54.374631 23485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:54.376080 23485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:54.377489 23485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:54.391814 23491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:54.391956 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:54.391918 23492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:54.393100 23485 server_base.cc:1034] running on GCE node
I20260806 08:56:54.393816 23485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:54.394910 23485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:54.396173 23485 hybrid_clock.cc:648] HybridClock initialized: now 1786006614396114 us; error 45 us; skew 500 ppm
I20260806 08:56:54.396559 23485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:54.400208 23485 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:56:54.402206 23485 webserver.cc:460] Webserver started at http://127.22.98.3:41649/ using document root <none> and password file <none>
I20260806 08:56:54.403028 23485 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:54.403163 23485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:54.403510 23485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:54.406976 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "546c035f3bad4ebcb1db655d5146fda9"
format_stamp: "Formatted at 2026-08-06 08:56:54 on dist-test-slave-jpj1"
server_key: "94e51875e3fff63b38e7ac942d0a83c9"
server_key_iv: "e6e8695403a5d8388423157a68d971e6"
server_key_version: "encryptionkey@0"
I20260806 08:56:54.407850 23485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "546c035f3bad4ebcb1db655d5146fda9"
format_stamp: "Formatted at 2026-08-06 08:56:54 on dist-test-slave-jpj1"
server_key: "94e51875e3fff63b38e7ac942d0a83c9"
server_key_iv: "e6e8695403a5d8388423157a68d971e6"
server_key_version: "encryptionkey@0"
I20260806 08:56:54.413496 23485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:56:54.417325 23503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:54.418799 23485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 08:56:54.418972 23485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "546c035f3bad4ebcb1db655d5146fda9"
format_stamp: "Formatted at 2026-08-06 08:56:54 on dist-test-slave-jpj1"
server_key: "94e51875e3fff63b38e7ac942d0a83c9"
server_key_iv: "e6e8695403a5d8388423157a68d971e6"
server_key_version: "encryptionkey@0"
I20260806 08:56:54.419162 23485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:54.435942 23485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:54.440090 23485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:54.440363 23485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:54.441394 23485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:54.442935 23485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:54.443013 23485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:54.443135 23485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:54.443193 23485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:54.485831 23485 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:38249
I20260806 08:56:54.485863 23660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:38249 every 8 connection(s)
I20260806 08:56:54.487092 23485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:56:54.494763 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 23485
I20260806 08:56:54.494937 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:56:54.495364 22920 external_mini_cluster.cc:1428] Setting key becf325fc9d5dc1112cd86be0720a9e3
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:54.508584 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.489645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59883 (local address 127.22.98.62:42515)
0806 08:56:54.490132 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.490136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.491116 (+ 980us) server_negotiation.cc:407] Connection header received
0806 08:56:54.493012 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.493018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.493125 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.493221 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.494794 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.495735 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.497741 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.498288 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.498417 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.502465 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.502470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.502474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.502510 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.505218 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.505940 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.505945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.505948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.506111 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.506501 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.506504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.506506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.506814 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.506933 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.507778 (+ 845us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.508006 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":240,"threads_started":1}
I20260806 08:56:54.509598 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.490055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42515 (local address 127.22.98.3:59883)
0806 08:56:54.490928 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:54.490977 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:54.492729 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:54.493485 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:54.493494 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:54.494019 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:54.494616 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:54.494635 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.495908 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.495912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:54.497599 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:54.497614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.498525 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.498530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:54.498873 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.499651 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:54.499676 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:54.502237 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:54.505349 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:54.505356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:54.505369 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:54.505803 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:54.506221 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:54.506225 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:54.506227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:54.506420 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:54.506992 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:54.506998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:54.507363 (+ 365us) client_negotiation.cc:769] Sending connection context
0806 08:56:54.507697 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 08:56:54.508310 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":390,"threads_started":1}
I20260806 08:56:54.511339 23661 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42515
I20260806 08:56:54.511695 23661 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:54.512615 23661 heartbeater.cc:511] Master 127.22.98.62:42515 requested a full tablet report, sending...
I20260806 08:56:54.514074 22995 ts_manager.cc:194] Registered new tserver with Master: 546c035f3bad4ebcb1db655d5146fda9 (127.22.98.3:38249)
I20260806 08:56:54.514896 22995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:59883
I20260806 08:56:54.525260 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:56:54.578356 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.560930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.22.98.1:36721)
0806 08:56:54.561942 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.561952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.561984 (+ 32us) server_negotiation.cc:407] Connection header received
0806 08:56:54.562054 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.562061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.562363 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.562582 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.564121 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.565303 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.566784 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.567608 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.567734 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.572367 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.572379 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.572396 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.572434 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.575522 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.576398 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.576406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.576409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.576546 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.576948 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.576951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.576953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.577321 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.577551 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.577829 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.578045 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":652,"threads_started":1}
W20260806 08:56:54.580119 23137 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52600
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:56:54.629665 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.615455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.22.98.1:36721)
0806 08:56:54.615729 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.615736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.615847 (+ 111us) server_negotiation.cc:407] Connection header received
0806 08:56:54.616003 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.616007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.616106 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.616193 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.617640 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.618363 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.619708 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.620167 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.620268 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.624820 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.624827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.624830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.624866 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.627422 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.628110 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.628115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.628117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.628257 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.628603 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.628606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.628608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.628848 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.629022 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.629247 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.629410 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 08:56:54.631762 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 23075
I20260806 08:56:54.644500 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.632907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.22.98.1:36721)
0806 08:56:54.633059 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.633063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.633320 (+ 257us) server_negotiation.cc:407] Connection header received
0806 08:56:54.633434 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.633438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.633491 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.633565 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.634893 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.635644 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.636939 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.637364 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.637453 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.639649 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.639655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.639659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.639692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.642303 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.642908 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.642911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.642914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.643050 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.643365 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.643368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.643370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.643651 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.643833 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.644068 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.644217 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 08:56:54.645391 23265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:54.734537 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 23075
I20260806 08:56:54.757330 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 23285
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:56:54.773342 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.758261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39246 (local address 127.22.98.2:43091)
0806 08:56:54.758802 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.758809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.758853 (+ 44us) server_negotiation.cc:407] Connection header received
0806 08:56:54.758922 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.758929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.759138 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.759277 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.760415 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.761397 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.762672 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.763283 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.763390 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.767582 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.767599 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.767619 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.767663 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.770537 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.771172 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.771179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.771181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.771315 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.771886 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.771891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.771893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.772254 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.772513 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.772841 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.773048 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":268,"threads_started":1}
I20260806 08:56:54.774670 23469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:54.856446 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 23285
I20260806 08:56:54.878926 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 23485
Aug 06 08:56:54 dist-test-slave-jpj1 krb5kdc[22929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:56:54.895054 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:54.879887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.22.98.3:38249)
0806 08:56:54.880512 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:54.880520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:54.880551 (+ 31us) server_negotiation.cc:407] Connection header received
0806 08:56:54.880621 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:54.880627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:54.880864 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:54.881018 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:54.882259 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.883328 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.884657 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:54.885229 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:54.885354 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:54.889382 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:54.889394 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:54.889410 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:54.889444 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:54.892292 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.892954 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.892964 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.892966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.893096 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:54.893474 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:54.893478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:54.893480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:54.893845 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:54.894149 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:54.894453 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 08:56:54.894720 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":287,"threads_started":1}
I20260806 08:56:54.896435 23655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:54.978332 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 23485
I20260806 08:56:54.997203 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22949
I20260806 08:56:54.998483 23036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:55.102504 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22949
2026-08-06T08:56:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4681 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:56:55 dist-test-slave-jpj1 krb5kdc[23694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:56:55 dist-test-slave-jpj1 krb5kdc[23694](info): set up 2 sockets
Aug 06 08:56:55 dist-test-slave-jpj1 krb5kdc[23694](info): commencing operation
krb5kdc: starting...
W20260806 08:56:56.279170 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:56:56 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:56:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:56:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:56:56.475032 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:33613
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:33613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:56.802174 23712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:56.802477 23712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:56.802579 23712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:56.809773 23712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:56:56.809894 23712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:56.809942 23712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:56.810010 23712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:56:56.810061 23712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:56:56.819973 23712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:33613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:56.821331 23712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:56.822661 23712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:56.834245 23725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:56.834334 23719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:56.834334 23723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:56.835201 23712 server_base.cc:1034] running on GCE node
I20260806 08:56:56.836079 23712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:56.837452 23712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:56.838706 23712 hybrid_clock.cc:648] HybridClock initialized: now 1786006616838647 us; error 44 us; skew 500 ppm
I20260806 08:56:56.839048 23712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:56 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:56.842200 23712 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:56:56.844116 23712 webserver.cc:460] Webserver started at http://127.22.98.62:38675/ using document root <none> and password file <none>
I20260806 08:56:56.844846 23712 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:56.844969 23712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:56.845299 23712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:56.848555 23712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "c9de0c23521d4b0ca798a0aed41d84a2"
format_stamp: "Formatted at 2026-08-06 08:56:56 on dist-test-slave-jpj1"
server_key: "767cda29067adde7c06c7ccc9db15883"
server_key_iv: "fc3ca41c72345a5dd141e3e275e1f6f8"
server_key_version: "encryptionkey@0"
I20260806 08:56:56.849336 23712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "c9de0c23521d4b0ca798a0aed41d84a2"
format_stamp: "Formatted at 2026-08-06 08:56:56 on dist-test-slave-jpj1"
server_key: "767cda29067adde7c06c7ccc9db15883"
server_key_iv: "fc3ca41c72345a5dd141e3e275e1f6f8"
server_key_version: "encryptionkey@0"
I20260806 08:56:56.854708 23712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:56:56.858297 23736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:56.859712 23712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 08:56:56.859876 23712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "c9de0c23521d4b0ca798a0aed41d84a2"
format_stamp: "Formatted at 2026-08-06 08:56:56 on dist-test-slave-jpj1"
server_key: "767cda29067adde7c06c7ccc9db15883"
server_key_iv: "fc3ca41c72345a5dd141e3e275e1f6f8"
server_key_version: "encryptionkey@0"
I20260806 08:56:56.860056 23712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:56.882691 23712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:56.886857 23712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:56.887143 23712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:56.905391 23712 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:33613
I20260806 08:56:56.905414 23807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:33613 every 8 connection(s)
I20260806 08:56:56.906847 23712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:56:56.910820 23809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:56:56.914003 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 23712
I20260806 08:56:56.914211 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:56:56.914839 22920 external_mini_cluster.cc:1428] Setting key 5c56f0032c50f7cdea4656e6b79b72a9
I20260806 08:56:56.923031 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2: Bootstrap starting.
I20260806 08:56:56.926303 23809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2: Neither blocks nor log segments found. Creating new log.
I20260806 08:56:56.927335 23809 log.cc:824] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2: Log is configured to *not* fsync() on all Append() calls
Aug 06 08:56:56 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:56.930480 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2: No bootstrap required, opened a new log
I20260806 08:56:56.936837 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:56.916912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.22.98.62:33613)
0806 08:56:56.917719 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:56.917733 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:56.917787 (+ 54us) server_negotiation.cc:407] Connection header received
0806 08:56:56.919485 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:56.919510 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:56.920015 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:56.920420 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:56.921949 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:56.923329 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:56.924583 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:56.925267 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:56.925431 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:56.929643 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:56.929652 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:56.929668 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:56.929703 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:56.932714 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:56.933504 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:56.933511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:56.933517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:56.933674 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:56.934061 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:56.934065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:56.934067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:56.934592 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:56.934817 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:56.935361 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 08:56:56.935822 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":358,"threads_started":1}
I20260806 08:56:56.939282 23809 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } }
I20260806 08:56:56.939704 23809 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:56:56.939821 23809 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9de0c23521d4b0ca798a0aed41d84a2, State: Initialized, Role: FOLLOWER
I20260806 08:56:56.940372 23809 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } }
I20260806 08:56:56.940594 23809 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:56:56.940732 23809 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:56:56.940893 23809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:56:56.942992 23809 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } }
I20260806 08:56:56.943517 23809 leader_election.cc:302] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9de0c23521d4b0ca798a0aed41d84a2; no voters:
I20260806 08:56:56.944175 23809 leader_election.cc:288] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:56:56.944306 23816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:56:56.944957 23816 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [term 1 LEADER]: Becoming Leader. State: Replica: c9de0c23521d4b0ca798a0aed41d84a2, State: Running, Role: LEADER
I20260806 08:56:56.945482 23816 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } }
I20260806 08:56:56.946193 23809 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:56:56.948357 23817 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } } }
I20260806 08:56:56.948735 23817 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [sys.catalog]: This master's current role is: LEADER
I20260806 08:56:56.949206 23818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9de0c23521d4b0ca798a0aed41d84a2. Latest consensus state: current_term: 1 leader_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9de0c23521d4b0ca798a0aed41d84a2" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 33613 } } }
I20260806 08:56:56.949518 23818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2 [sys.catalog]: This master's current role is: LEADER
I20260806 08:56:56.950873 23825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:56:56.955395 23825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:56:56.963909 23825 catalog_manager.cc:1347] Generated new cluster ID: b616092060e4429b8d0be63c6803ff9e
I20260806 08:56:56.964033 23825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:56:56.977669 23825 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:56:56.979120 23825 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:56:56.994364 23825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9de0c23521d4b0ca798a0aed41d84a2: Generated new TSK 0
I20260806 08:56:56.995399 23825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:56:57.045549 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:57.389850 23844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:57.390162 23844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:57.390307 23844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:57.397768 23844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:57.397910 23844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:57.398068 23844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:56:57.408950 23844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:57.410328 23844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:57.411859 23844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:57.425618 23856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:57.425596 23852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:57.425596 23853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:57.426299 23844 server_base.cc:1034] running on GCE node
I20260806 08:56:57.427013 23844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:57.428138 23844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:57.429373 23844 hybrid_clock.cc:648] HybridClock initialized: now 1786006617429305 us; error 58 us; skew 500 ppm
I20260806 08:56:57.429893 23844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:57 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:57.437523 23844 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:56:57.439162 23844 webserver.cc:460] Webserver started at http://127.22.98.1:42783/ using document root <none> and password file <none>
I20260806 08:56:57.439990 23844 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:57.440131 23844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:57.440485 23844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:57.443779 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "447394e6ebfc4f77bf216462f1781823"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "f3ee6d780df31064fdd8c5a66dde5154"
server_key_iv: "df3fd8328cef584be8511e2c33054982"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.444617 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "447394e6ebfc4f77bf216462f1781823"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "f3ee6d780df31064fdd8c5a66dde5154"
server_key_iv: "df3fd8328cef584be8511e2c33054982"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.451275 23844 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 08:56:57.455106 23865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:57.456729 23844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:56:57.456893 23844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "447394e6ebfc4f77bf216462f1781823"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "f3ee6d780df31064fdd8c5a66dde5154"
server_key_iv: "df3fd8328cef584be8511e2c33054982"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.457110 23844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:57.471967 23844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:57.477421 23844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:57.477779 23844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:57.479105 23844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:57.480814 23844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:57.480894 23844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:57.480988 23844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:57.481033 23844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:57.525388 23844 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:42203
I20260806 08:56:57.525457 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:42203 every 8 connection(s)
I20260806 08:56:57.526680 23844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:56:57.529244 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 23844
I20260806 08:56:57.529457 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:56:57.530422 22920 external_mini_cluster.cc:1428] Setting key d9c4475227d93a4ed7f2ef8c47f47b7e
Aug 06 08:56:57 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:57.552886 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:57.529935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:36271 (local address 127.22.98.62:33613)
0806 08:56:57.530500 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:57.530507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:57.531623 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 08:56:57.533596 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:57.533602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:57.533690 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:57.533789 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:57.535498 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:57.536642 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:57.539051 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:57.539760 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:57.539914 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:57.544242 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:57.544248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:57.544251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:57.544288 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:57.548429 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:57.549705 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:57.549711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:57.549714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:57.549931 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:57.550560 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:57.550565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:57.550568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:57.550964 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:57.551079 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:57.552195 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0806 08:56:57.552493 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":255,"threads_started":1}
I20260806 08:56:57.555359 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:57.530592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:33613 (local address 127.22.98.1:36271)
0806 08:56:57.531424 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:57.531475 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:57.533313 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:57.534058 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:57.534067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:57.534669 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:57.535313 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:57.535334 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:57.536861 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:57.536866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:57.538881 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:57.538900 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:57.540027 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:57.540035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:57.540332 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:57.541116 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:57.541144 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:57.543993 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:57.548683 (+ 4690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:57.548693 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:57.548744 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:57.549523 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:57.550118 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:57.550123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:57.550126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:57.550431 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:57.551177 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:57.551185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:57.551684 (+ 499us) client_negotiation.cc:769] Sending connection context
0806 08:56:57.553129 (+ 1445us) client_negotiation.cc:240] Negotiation successful
0806 08:56:57.553767 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":338,"threads_started":1}
I20260806 08:56:57.557293 24027 heartbeater.cc:347] Connected to a master server at 127.22.98.62:33613
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
I20260806 08:56:57.557646 24027 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:57.558483 24027 heartbeater.cc:511] Master 127.22.98.62:33613 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
I20260806 08:56:57.561136 23759 ts_manager.cc:194] Registered new tserver with Master: 447394e6ebfc4f77bf216462f1781823 (127.22.98.1:42203)
I20260806 08:56:57.562669 23759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:36271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:56:57.586599 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:57.911456 24037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:57.911770 24037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:57.911914 24037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:57.919008 24037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:57.919127 24037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:57.919293 24037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:56:57.928984 24037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:57.930335 24037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:57.931751 24037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:57.944384 24046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:57.944520 24045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:57.944475 24050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:57.945751 24037 server_base.cc:1034] running on GCE node
I20260806 08:56:57.946368 24037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:57.947320 24037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:57.948515 24037 hybrid_clock.cc:648] HybridClock initialized: now 1786006617948450 us; error 54 us; skew 500 ppm
I20260806 08:56:57.948827 24037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:57 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:57.952092 24037 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:56:57.953483 24037 webserver.cc:460] Webserver started at http://127.22.98.2:39079/ using document root <none> and password file <none>
I20260806 08:56:57.954170 24037 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:57.954314 24037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:57.954627 24037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:57.957548 24037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "2717b93666f641098116a66539836115"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "50949d0616e0b9b93fc0ef6a0bbca213"
server_key_iv: "02af0cd2109299d594c07c54c9f8255c"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.958287 24037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "2717b93666f641098116a66539836115"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "50949d0616e0b9b93fc0ef6a0bbca213"
server_key_iv: "02af0cd2109299d594c07c54c9f8255c"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.963317 24037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:56:57.966826 24061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:57.968180 24037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:56:57.968330 24037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "2717b93666f641098116a66539836115"
format_stamp: "Formatted at 2026-08-06 08:56:57 on dist-test-slave-jpj1"
server_key: "50949d0616e0b9b93fc0ef6a0bbca213"
server_key_iv: "02af0cd2109299d594c07c54c9f8255c"
server_key_version: "encryptionkey@0"
I20260806 08:56:57.968518 24037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:57.998041 24037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:58.002130 24037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:58.002404 24037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:58.003412 24037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:58.004770 24037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:58.004843 24037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:58.004962 24037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:58.005012 24037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:58.035984 24037 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:32807
I20260806 08:56:58.036055 24208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:32807 every 8 connection(s)
I20260806 08:56:58.037257 24037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:56:58.045899 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 24037
I20260806 08:56:58.046079 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:56:58.046519 22920 external_mini_cluster.cc:1428] Setting key 7abeb72c3cca939315eac54021968839
Aug 06 08:56:58 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:58.057844 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.039770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:52041 (local address 127.22.98.62:33613)
0806 08:56:58.039952 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.039957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.041193 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 08:56:58.043055 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.043061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.043145 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.043233 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:58.044768 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.045558 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.047572 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.048110 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.048208 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.052195 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:58.052201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:58.052205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:58.052238 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:58.054694 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.055408 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.055414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.055417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.055577 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.056077 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.056083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.056087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.056428 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:58.056570 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.057263 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.057490 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":65}
I20260806 08:56:58.058653 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.040329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:33613 (local address 127.22.98.2:52041)
0806 08:56:58.041033 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.041070 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.042786 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.043478 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.043486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.044039 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:58.044599 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.044617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.045738 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.045746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.047397 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.047413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.048358 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.048366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.048652 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.049444 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:58.049475 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:58.051950 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:58.054830 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:58.054839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:58.054868 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:58.055277 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:58.055701 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:58.055706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:58.055710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:58.055932 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:58.056554 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:58.056560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:58.056891 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.057123 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.057516 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":286,"threads_started":1}
I20260806 08:56:58.060207 24209 heartbeater.cc:347] Connected to a master server at 127.22.98.62:33613
I20260806 08:56:58.060554 24209 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:58.061525 24209 heartbeater.cc:511] Master 127.22.98.62:33613 requested a full tablet report, sending...
I20260806 08:56:58.063232 23759 ts_manager.cc:194] Registered new tserver with Master: 2717b93666f641098116a66539836115 (127.22.98.2:32807)
I20260806 08:56:58.064131 23759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:52041
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:56:58.091194 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:56:58.426234 24215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:56:58.426540 24215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:56:58.426649 24215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:56:58.434787 24215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:56:58.434906 24215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:56:58.435020 24215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:56:58.445142 24215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:33613
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:56:58.446530 24215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:56:58.448040 24215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:56:58.461155 24222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:58.461233 24221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:56:58.461194 24227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:56:58.462458 24215 server_base.cc:1034] running on GCE node
I20260806 08:56:58.463135 24215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:56:58.464170 24215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:56:58.465476 24215 hybrid_clock.cc:648] HybridClock initialized: now 1786006618465397 us; error 61 us; skew 500 ppm
I20260806 08:56:58.465814 24215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:56:58 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006618, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:56:58.469059 24215 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:56:58.470507 24215 webserver.cc:460] Webserver started at http://127.22.98.3:39039/ using document root <none> and password file <none>
I20260806 08:56:58.471182 24215 fs_manager.cc:359] Metadata directory not provided
I20260806 08:56:58.471283 24215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:56:58.471580 24215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:56:58.474627 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "c1f7b5544f394f3095bba1cfd2b87469"
format_stamp: "Formatted at 2026-08-06 08:56:58 on dist-test-slave-jpj1"
server_key: "9461eaa60af050a0557d40de833fc2ca"
server_key_iv: "3cb348d2cbac99118b948ae54c500b6c"
server_key_version: "encryptionkey@0"
I20260806 08:56:58.475365 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "c1f7b5544f394f3095bba1cfd2b87469"
format_stamp: "Formatted at 2026-08-06 08:56:58 on dist-test-slave-jpj1"
server_key: "9461eaa60af050a0557d40de833fc2ca"
server_key_iv: "3cb348d2cbac99118b948ae54c500b6c"
server_key_version: "encryptionkey@0"
I20260806 08:56:58.480822 24215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:56:58.484895 24236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:58.486183 24215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:56:58.486346 24215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "c1f7b5544f394f3095bba1cfd2b87469"
format_stamp: "Formatted at 2026-08-06 08:56:58 on dist-test-slave-jpj1"
server_key: "9461eaa60af050a0557d40de833fc2ca"
server_key_iv: "3cb348d2cbac99118b948ae54c500b6c"
server_key_version: "encryptionkey@0"
I20260806 08:56:58.486536 24215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:56:58.508500 24215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:56:58.512979 24215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:56:58.513264 24215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:56:58.514334 24215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:56:58.515774 24215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:56:58.515861 24215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:58.515988 24215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:56:58.516044 24215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:56:58.552491 24215 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:35717
I20260806 08:56:58.552735 24402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:35717 every 8 connection(s)
I20260806 08:56:58.553857 24215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:56:58.561864 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 24215
I20260806 08:56:58.562111 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:56:58.562636 22920 external_mini_cluster.cc:1428] Setting key be4bc08c20da7a8a7f576af4a915e8e0
I20260806 08:56:58.566403 24027 heartbeater.cc:503] Master 127.22.98.62:33613 was elected leader, sending a full tablet report...
Aug 06 08:56:58 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006618, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:56:58.576763 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.556814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:41653 (local address 127.22.98.62:33613)
0806 08:56:58.557037 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.557044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.558419 (+ 1375us) server_negotiation.cc:407] Connection header received
0806 08:56:58.560333 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.560338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.560443 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.560523 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:58.562427 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.563259 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.565367 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.565892 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.566018 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.570597 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:58.570603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:58.570606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:58.570638 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:58.573580 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.574318 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.574322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.574324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.574469 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.574858 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.574861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.574863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.575123 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:58.575252 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.576191 (+ 939us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.576442 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 08:56:58.578028 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.557375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:33613 (local address 127.22.98.3:41653)
0806 08:56:58.558249 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.558296 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.560041 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.560746 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.560754 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.561308 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:56:58.562229 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.562269 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.563450 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.563455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.565226 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.565241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.566173 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.566182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.566468 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.567272 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:56:58.567302 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:56:58.570384 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:56:58.573719 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:58.573727 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:58.573769 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:58.574198 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:58.574554 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:56:58.574557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:56:58.574559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:56:58.574768 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:56:58.575310 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:56:58.575342 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:56:58.575771 (+ 429us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.576035 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.576519 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":374,"threads_started":1}
I20260806 08:56:58.579411 24403 heartbeater.cc:347] Connected to a master server at 127.22.98.62:33613
I20260806 08:56:58.579722 24403 heartbeater.cc:464] Registering TS with master...
I20260806 08:56:58.580420 24403 heartbeater.cc:511] Master 127.22.98.62:33613 requested a full tablet report, sending...
I20260806 08:56:58.581765 23759 ts_manager.cc:194] Registered new tserver with Master: c1f7b5544f394f3095bba1cfd2b87469 (127.22.98.3:35717)
I20260806 08:56:58.582562 23759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:41653
I20260806 08:56:58.590656 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 08:56:58.611074 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.598224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.22.98.62:33613)
0806 08:56:58.598397 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.598402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.599026 (+ 624us) server_negotiation.cc:407] Connection header received
0806 08:56:58.599164 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.599168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.599232 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.599324 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:58.600911 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.601722 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.603267 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.603763 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.603865 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.606224 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:58.606229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:58.606232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:58.606263 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:58.608701 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.609307 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.609311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.609314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.609443 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.609737 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.609740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.609742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.610063 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:58.610266 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.610524 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.610703 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:56:58.620368 23759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:56:58.629061 23759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:56:58.659062 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.646639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.22.98.2:32807)
0806 08:56:58.648595 (+ 1956us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.648605 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.649239 (+ 634us) server_negotiation.cc:407] Connection header received
0806 08:56:58.649379 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.649388 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.649727 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.649936 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:56:58.650986 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.652992 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.655930 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.657567 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.657683 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.657896 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.658452 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.658670 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1770,"thread_start_us":2082,"threads_started":1}
I20260806 08:56:58.659061 24427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.646529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:32807 (local address 127.0.0.1:44620)
0806 08:56:58.648982 (+ 2453us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.649000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.649118 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.650259 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.650268 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.650301 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:56:58.650757 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.650772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.653222 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.653229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.655728 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.655744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.658034 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.658039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.658201 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.658237 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.658374 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.658581 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2342,"thread_start_us":2215,"threads_started":1}
I20260806 08:56:58.663833 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.649027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42203 (local address 127.0.0.1:38308)
0806 08:56:58.655603 (+ 6576us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.655623 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.655771 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.656593 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.656598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.656627 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:56:58.657091 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.657105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.659351 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.659357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.661487 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.661500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.663074 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.663081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.663257 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.663289 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.663372 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.663507 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6399,"thread_start_us":6344,"threads_started":1}
I20260806 08:56:58.664464 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.648570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.22.98.1:42203)
0806 08:56:58.655339 (+ 6769us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.655348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.655918 (+ 570us) server_negotiation.cc:407] Connection header received
0806 08:56:58.656053 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.656063 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.656357 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.657283 (+ 926us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:56:58.657464 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.659197 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.661643 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.662859 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.662980 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.663225 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.664080 (+ 855us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.664234 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6586,"thread_start_us":6328,"threads_started":1}
I20260806 08:56:58.664685 24122 tablet_service.cc:1467] Processing CreateTablet for tablet c14f4b2d39484ee0b6d8f6d0c2cad540 (DEFAULT_TABLE table=test-table [id=bcb5a941d42f4550816886f1fcfa581a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:56:58.665510 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.646056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35717 (local address 127.0.0.1:41786)
0806 08:56:58.647062 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.647116 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.647370 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.657668 (+ 10298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.657674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.657703 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:56:58.658151 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.658167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.660852 (+ 2685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.660858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.663048 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.663066 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.664888 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.664893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.665043 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.665068 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.665138 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.665249 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":786,"threads_started":1}
I20260806 08:56:58.665680 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.646428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.22.98.3:35717)
0806 08:56:58.657015 (+ 10587us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.657023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.657063 (+ 40us) server_negotiation.cc:407] Connection header received
0806 08:56:58.657166 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.657174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.657438 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.657669 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:56:58.658358 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.660717 (+ 2359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.663266 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.664688 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.664820 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.665044 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.665179 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.665336 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10376,"thread_start_us":10226,"threads_started":1}
I20260806 08:56:58.666478 24122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14f4b2d39484ee0b6d8f6d0c2cad540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:56:58.671200 23934 tablet_service.cc:1467] Processing CreateTablet for tablet c14f4b2d39484ee0b6d8f6d0c2cad540 (DEFAULT_TABLE table=test-table [id=bcb5a941d42f4550816886f1fcfa581a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:56:58.671234 24309 tablet_service.cc:1467] Processing CreateTablet for tablet c14f4b2d39484ee0b6d8f6d0c2cad540 (DEFAULT_TABLE table=test-table [id=bcb5a941d42f4550816886f1fcfa581a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:56:58.672596 23934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14f4b2d39484ee0b6d8f6d0c2cad540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:56:58.672685 24309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14f4b2d39484ee0b6d8f6d0c2cad540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:56:58.684621 24431 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Bootstrap starting.
I20260806 08:56:58.686527 24432 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Bootstrap starting.
I20260806 08:56:58.688800 24433 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Bootstrap starting.
I20260806 08:56:58.690423 24432 tablet_bootstrap.cc:654] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Neither blocks nor log segments found. Creating new log.
I20260806 08:56:58.690425 24431 tablet_bootstrap.cc:654] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Neither blocks nor log segments found. Creating new log.
I20260806 08:56:58.691639 24432 log.cc:824] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Log is configured to *not* fsync() on all Append() calls
I20260806 08:56:58.691982 24431 log.cc:824] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Log is configured to *not* fsync() on all Append() calls
I20260806 08:56:58.692785 24433 tablet_bootstrap.cc:654] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Neither blocks nor log segments found. Creating new log.
I20260806 08:56:58.694301 24433 log.cc:824] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Log is configured to *not* fsync() on all Append() calls
I20260806 08:56:58.694864 24431 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: No bootstrap required, opened a new log
I20260806 08:56:58.695086 24432 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: No bootstrap required, opened a new log
I20260806 08:56:58.695158 24431 ts_tablet_manager.cc:1397] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 08:56:58.695355 24432 ts_tablet_manager.cc:1397] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 08:56:58.697727 24433 tablet_bootstrap.cc:492] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: No bootstrap required, opened a new log
I20260806 08:56:58.698076 24433 ts_tablet_manager.cc:1397] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 08:56:58.703491 24432 raft_consensus.cc:348] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.703907 24432 raft_consensus.cc:374] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:56:58.704011 24432 raft_consensus.cc:729] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447394e6ebfc4f77bf216462f1781823, State: Initialized, Role: FOLLOWER
I20260806 08:56:58.704643 24432 consensus_queue.cc:260] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.706683 24432 ts_tablet_manager.cc:1428] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260806 08:56:58.707489 24431 raft_consensus.cc:348] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.708027 24431 raft_consensus.cc:374] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:56:58.708137 24431 raft_consensus.cc:729] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2717b93666f641098116a66539836115, State: Initialized, Role: FOLLOWER
I20260806 08:56:58.709000 24431 consensus_queue.cc:260] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.710287 24209 heartbeater.cc:503] Master 127.22.98.62:33613 was elected leader, sending a full tablet report...
I20260806 08:56:58.710119 24433 raft_consensus.cc:348] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.710572 24433 raft_consensus.cc:374] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:56:58.710686 24433 raft_consensus.cc:729] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f7b5544f394f3095bba1cfd2b87469, State: Initialized, Role: FOLLOWER
I20260806 08:56:58.711695 24431 ts_tablet_manager.cc:1428] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260806 08:56:58.711947 24433 consensus_queue.cc:260] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.713214 24403 heartbeater.cc:503] Master 127.22.98.62:33613 was elected leader, sending a full tablet report...
I20260806 08:56:58.713968 24433 ts_tablet_manager.cc:1428] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
W20260806 08:56:58.779250 24028 tablet.cc:2369] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:56:58.789304 24210 tablet.cc:2369] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:56:58.806389 24404 tablet.cc:2369] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:56:58.888751 24440 raft_consensus.cc:482] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:56:58.889106 24440 raft_consensus.cc:504] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.891422 24440 leader_election.cc:288] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1f7b5544f394f3095bba1cfd2b87469 (127.22.98.3:35717), 447394e6ebfc4f77bf216462f1781823 (127.22.98.1:42203)
I20260806 08:56:58.899041 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.891836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:55813 (local address 127.22.98.3:35717)
0806 08:56:58.892118 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.892124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.892497 (+ 373us) server_negotiation.cc:407] Connection header received
0806 08:56:58.892725 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.892730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.892824 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.892926 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:56:58.893891 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.894873 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.897128 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.898033 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.898130 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.898207 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.898643 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.898784 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 08:56:58.899163 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.891760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35717 (local address 127.22.98.2:55813)
0806 08:56:58.892357 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.892374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.892568 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.893163 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.893168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.893198 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:56:58.893729 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.893747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.895032 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.895038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.896969 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.896983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.898264 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.898272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.898447 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.898482 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.898588 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.898726 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":1,"thread_start_us":290,"threads_started":1}
I20260806 08:56:58.899714 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.892835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:46917 (local address 127.22.98.1:42203)
0806 08:56:58.893055 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.893061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.893530 (+ 469us) server_negotiation.cc:407] Connection header received
0806 08:56:58.893819 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.893825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.893915 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.893993 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:56:58.894507 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.895647 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.897508 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.898571 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.898694 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.898800 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.899309 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.899442 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 08:56:58.900399 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.892511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42203 (local address 127.22.98.2:46917)
0806 08:56:58.893433 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 08:56:58.893451 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:56:58.893644 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:56:58.894025 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:56:58.894029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:56:58.894053 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:56:58.894350 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.894359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.895824 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.895828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.897360 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:56:58.897374 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.898814 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.898820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:56:58.899078 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.899115 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:56:58.899942 (+ 827us) client_negotiation.cc:240] Negotiation successful
0806 08:56:58.900037 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":22,"thread_start_us":520,"threads_started":1}
I20260806 08:56:58.900300 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14f4b2d39484ee0b6d8f6d0c2cad540" candidate_uuid: "2717b93666f641098116a66539836115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f7b5544f394f3095bba1cfd2b87469" is_pre_election: true
I20260806 08:56:58.901393 24338 raft_consensus.cc:2454] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2717b93666f641098116a66539836115 in term 0.
I20260806 08:56:58.901476 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14f4b2d39484ee0b6d8f6d0c2cad540" candidate_uuid: "2717b93666f641098116a66539836115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447394e6ebfc4f77bf216462f1781823" is_pre_election: true
I20260806 08:56:58.902026 23960 raft_consensus.cc:2454] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2717b93666f641098116a66539836115 in term 0.
I20260806 08:56:58.902573 24062 leader_election.cc:302] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2717b93666f641098116a66539836115, c1f7b5544f394f3095bba1cfd2b87469; no voters:
I20260806 08:56:58.903026 24440 raft_consensus.cc:2781] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:56:58.903193 24440 raft_consensus.cc:482] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:56:58.903352 24440 raft_consensus.cc:3037] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:56:58.905673 24440 raft_consensus.cc:504] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.906257 24440 leader_election.cc:288] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [CANDIDATE]: Term 1 election: Requested vote from peers c1f7b5544f394f3095bba1cfd2b87469 (127.22.98.3:35717), 447394e6ebfc4f77bf216462f1781823 (127.22.98.1:42203)
I20260806 08:56:58.906706 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14f4b2d39484ee0b6d8f6d0c2cad540" candidate_uuid: "2717b93666f641098116a66539836115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "447394e6ebfc4f77bf216462f1781823"
I20260806 08:56:58.906774 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14f4b2d39484ee0b6d8f6d0c2cad540" candidate_uuid: "2717b93666f641098116a66539836115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f7b5544f394f3095bba1cfd2b87469"
I20260806 08:56:58.906863 23960 raft_consensus.cc:3037] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:56:58.906970 24338 raft_consensus.cc:3037] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:56:58.909281 23960 raft_consensus.cc:2454] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 447394e6ebfc4f77bf216462f1781823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2717b93666f641098116a66539836115 in term 1.
I20260806 08:56:58.909655 24062 leader_election.cc:302] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2717b93666f641098116a66539836115, 447394e6ebfc4f77bf216462f1781823; no voters:
I20260806 08:56:58.909930 24440 raft_consensus.cc:2781] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:56:58.910068 24338 raft_consensus.cc:2454] T c14f4b2d39484ee0b6d8f6d0c2cad540 P c1f7b5544f394f3095bba1cfd2b87469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2717b93666f641098116a66539836115 in term 1.
I20260806 08:56:58.910638 24440 raft_consensus.cc:686] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [term 1 LEADER]: Becoming Leader. State: Replica: 2717b93666f641098116a66539836115, State: Running, Role: LEADER
I20260806 08:56:58.911129 24440 consensus_queue.cc:237] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } }
I20260806 08:56:58.917920 23758 catalog_manager.cc:5495] T c14f4b2d39484ee0b6d8f6d0c2cad540 P 2717b93666f641098116a66539836115 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2717b93666f641098116a66539836115 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "2717b93666f641098116a66539836115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1f7b5544f394f3095bba1cfd2b87469" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2717b93666f641098116a66539836115" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 32807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "447394e6ebfc4f77bf216462f1781823" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42203 } health_report { overall_health: UNKNOWN } } }
Aug 06 08:56:58 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:56:58.970820 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:58.954524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.22.98.1:42203)
0806 08:56:58.954734 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:58.954741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:58.955455 (+ 714us) server_negotiation.cc:407] Connection header received
0806 08:56:58.955648 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:58.955652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:58.955781 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:58.955876 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:58.957629 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.958430 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.960072 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:58.960496 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:58.960617 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:58.964888 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:58.964902 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:58.964920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:58.964959 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:58.968034 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.968711 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.968719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.968721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.968863 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:58.969241 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:58.969250 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:58.969252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:58.969649 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:58.969943 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:58.970267 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 08:56:58.970485 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 08:56:58 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:56:59.024183 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:59.008862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.22.98.1:42203)
0806 08:56:59.009080 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:59.009085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:59.009283 (+ 198us) server_negotiation.cc:407] Connection header received
0806 08:56:59.009443 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:59.009447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:59.009512 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:59.009602 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:59.011232 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.012062 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.013463 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.013884 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.013980 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:59.018282 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:59.018296 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:59.018299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:59.018341 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:59.021191 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.021970 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.021975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.021977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.022197 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.022760 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.022765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.022767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.023168 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:59.023412 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:59.023669 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 08:56:59.023858 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[23694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:56:59.077220 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:59.061404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.22.98.1:42203)
0806 08:56:59.061637 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:59.061644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:59.062157 (+ 513us) server_negotiation.cc:407] Connection header received
0806 08:56:59.062359 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:59.062363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:59.062431 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:59.062512 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:59.064131 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.064856 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.066200 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.066793 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.066918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:59.071331 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:59.071337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:59.071340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:59.071380 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:59.074287 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.075075 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.075080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.075082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.075213 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.075594 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.075599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.075602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.075924 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:59.076171 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:59.076563 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 08:56:59.076871 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 08:56:59.079880 23908 tablet_service.cc:2908] Rejecting scan request for tablet c14f4b2d39484ee0b6d8f6d0c2cad540: Uninitialized: clean time has not yet been initialized
I20260806 08:56:59.084335 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 23844
I20260806 08:56:59.098114 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:59.085281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.22.98.1:42203)
0806 08:56:59.085436 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:59.085440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:59.085838 (+ 398us) server_negotiation.cc:407] Connection header received
0806 08:56:59.085969 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:59.085972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:59.086048 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:59.086131 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:59.087335 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.088064 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.089329 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.089796 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.089897 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:59.092506 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:59.092512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:59.092515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:59.092549 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:59.095461 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.096220 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.096224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.096227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.096361 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.096744 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.096748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.096750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.097153 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:59.097424 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:59.097673 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 08:56:59.097857 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 08:56:59.098839 24020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:59.188452 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 23844
I20260806 08:56:59.212657 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24037
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:56:59.228487 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:59.213496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.22.98.2:32807)
0806 08:56:59.213975 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:59.213979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:59.213996 (+ 17us) server_negotiation.cc:407] Connection header received
0806 08:56:59.214052 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:59.214055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:59.214139 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:59.214252 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:59.215608 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.216313 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.217686 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.218135 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.218257 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:59.222475 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:59.222492 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:59.222511 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:59.222551 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:59.225466 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.226206 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.226215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.226218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.226376 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.226763 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.226767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.226769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.227121 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:59.227426 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:59.227903 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 08:56:59.228126 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260806 08:56:59.229180 24201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:59.319006 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 24037
I20260806 08:56:59.340085 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24215
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[23694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:56:59.355773 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:56:59.340950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.22.98.3:35717)
0806 08:56:59.341143 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 08:56:59.341149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:56:59.341171 (+ 22us) server_negotiation.cc:407] Connection header received
0806 08:56:59.341316 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:56:59.341321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:56:59.341400 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:56:59.341489 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:56:59.342774 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.343686 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.345012 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:56:59.345415 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:56:59.345530 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:56:59.349881 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:56:59.349897 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:56:59.349918 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:56:59.349955 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:56:59.353058 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.353778 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.353788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.353792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.353955 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:56:59.354321 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:56:59.354324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:56:59.354327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:56:59.354706 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:56:59.354952 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 08:56:59.355172 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 08:56:59.355381 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:56:59.356467 24395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:59.447237 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 24215
I20260806 08:56:59.466987 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 23712
I20260806 08:56:59.468293 23801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:56:59.578584 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 23712
2026-08-06T08:56:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4474 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[24478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[24478](info): set up 2 sockets
Aug 06 08:56:59 dist-test-slave-jpj1 krb5kdc[24478](info): commencing operation
krb5kdc: starting...
W20260806 08:57:00.747009 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:00 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:00.951354 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36143
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:36143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:01.284886 24498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:01.285178 24498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:01.285312 24498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:01.292526 24498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:01.292683 24498 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 08:57:01.292757 24498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:01.292845 24498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:01.292910 24498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:01.292999 24498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:01.303171 24498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:36143
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:01.304539 24498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:01.305924 24498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:01.317675 24503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:01.317703 24504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:01.317633 24506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:01.319250 24498 server_base.cc:1034] running on GCE node
I20260806 08:57:01.320060 24498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:01.321480 24498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:01.322762 24498 hybrid_clock.cc:648] HybridClock initialized: now 1786006621322697 us; error 29 us; skew 500 ppm
I20260806 08:57:01.323141 24498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:01 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:01.326445 24498 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:01.327989 24498 webserver.cc:460] Webserver started at http://127.22.98.62:36963/ using document root <none> and password file <none>
I20260806 08:57:01.328706 24498 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:01.328797 24498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:01.329119 24498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:01.332223 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "9463c8114a49403dbc3826cb36e6ca88"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "1d9f758638170b0fb1d6f9e8ca80841c"
server_key_iv: "de87f722fd68dd782e957f0b9090178f"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.333094 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "9463c8114a49403dbc3826cb36e6ca88"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "1d9f758638170b0fb1d6f9e8ca80841c"
server_key_iv: "de87f722fd68dd782e957f0b9090178f"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.338771 24498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:01.342638 24517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:01.344234 24498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:01.344417 24498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "9463c8114a49403dbc3826cb36e6ca88"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "1d9f758638170b0fb1d6f9e8ca80841c"
server_key_iv: "de87f722fd68dd782e957f0b9090178f"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.344647 24498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:01.365186 24498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:01.369374 24498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:01.369769 24498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:01.391296 24498 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:36143
I20260806 08:57:01.391304 24615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:36143 every 8 connection(s)
I20260806 08:57:01.392748 24498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:01.396497 24617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:01.402078 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 24498
I20260806 08:57:01.402309 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:01.403118 22920 external_mini_cluster.cc:1428] Setting key 37b55fac123d21259bfcd3c2e0aaae36
I20260806 08:57:01.408671 24617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Bootstrap starting.
I20260806 08:57:01.412410 24617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:01.413407 24617 log.cc:824] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:01.416226 24617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: No bootstrap required, opened a new log
Aug 06 08:57:01 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:01.424361 24617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } }
I20260806 08:57:01.424738 24617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:01.424825 24617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9463c8114a49403dbc3826cb36e6ca88, State: Initialized, Role: FOLLOWER
I20260806 08:57:01.425462 24617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } }
I20260806 08:57:01.425681 24617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:01.425810 24617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:01.425980 24617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:01.428467 24617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } }
I20260806 08:57:01.428812 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:01.407734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.22.98.62:36143)
0806 08:57:01.408802 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:01.408831 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:01.408890 (+ 59us) server_negotiation.cc:407] Connection header received
0806 08:57:01.411176 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:01.411202 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:01.411650 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:01.412032 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:01.413494 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.414901 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.416180 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.416938 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.417129 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:01.421357 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:01.421369 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:01.421385 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:01.421430 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:01.424470 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:01.425263 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:01.425270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:01.425277 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:01.425456 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:01.425856 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:01.425861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:01.425864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:01.426534 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:01.426859 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:01.427324 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 08:57:01.427770 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":477,"threads_started":1}
I20260806 08:57:01.428989 24617 leader_election.cc:302] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9463c8114a49403dbc3826cb36e6ca88; no voters:
I20260806 08:57:01.429668 24617 leader_election.cc:288] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:01.429766 24622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:01.431905 24617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:01.432193 24622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [term 1 LEADER]: Becoming Leader. State: Replica: 9463c8114a49403dbc3826cb36e6ca88, State: Running, Role: LEADER
I20260806 08:57:01.432859 24622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } }
I20260806 08:57:01.436152 24624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9463c8114a49403dbc3826cb36e6ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } } }
I20260806 08:57:01.436583 24624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:01.437002 24625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9463c8114a49403dbc3826cb36e6ca88. Latest consensus state: current_term: 1 leader_uuid: "9463c8114a49403dbc3826cb36e6ca88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9463c8114a49403dbc3826cb36e6ca88" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36143 } } }
I20260806 08:57:01.437263 24625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:01.438374 24640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:01.443578 24640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:01.452566 24640 catalog_manager.cc:1347] Generated new cluster ID: 788de964c3f741db85e387ca58f6ce02
I20260806 08:57:01.452683 24640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:01.478824 24640 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:01.480226 24640 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:01.488035 24640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Generated new TSK 0
I20260806 08:57:01.489163 24640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:01.539120 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36143
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:01.871250 24649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:01.871605 24649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:01.871737 24649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:01.879179 24649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:01.879311 24649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:01.879397 24649 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:01.879480 24649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:01.890182 24649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:36143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:01.891465 24649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:01.892869 24649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:01.905534 24658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:01.905556 24660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:01.905493 24662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:01.906471 24649 server_base.cc:1034] running on GCE node
I20260806 08:57:01.907152 24649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:01.908161 24649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:01.909410 24649 hybrid_clock.cc:648] HybridClock initialized: now 1786006621909338 us; error 39 us; skew 500 ppm
I20260806 08:57:01.909775 24649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:01 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:01.913035 24649 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:01.914608 24649 webserver.cc:460] Webserver started at http://127.22.98.1:33151/ using document root <none> and password file <none>
I20260806 08:57:01.915370 24649 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:01.915462 24649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:01.915827 24649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:01.918915 24649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "0d516c77473d48158470cd8773c751af"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "0cce95cae2854cbffe320596b773e0cf"
server_key_iv: "662ab4ad73f5facafcddb50393af6ee9"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.919782 24649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "0d516c77473d48158470cd8773c751af"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "0cce95cae2854cbffe320596b773e0cf"
server_key_iv: "662ab4ad73f5facafcddb50393af6ee9"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.925346 24649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:57:01.929167 24670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:01.930814 24649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:01.930989 24649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "0d516c77473d48158470cd8773c751af"
format_stamp: "Formatted at 2026-08-06 08:57:01 on dist-test-slave-jpj1"
server_key: "0cce95cae2854cbffe320596b773e0cf"
server_key_iv: "662ab4ad73f5facafcddb50393af6ee9"
server_key_version: "encryptionkey@0"
I20260806 08:57:01.931214 24649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:01.955339 24649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:01.960034 24649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:01.960309 24649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:01.963270 24649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:01.963357 24649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:01.963491 24649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:01.963568 24649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:01 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:01.987224 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:01.964606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.1:43715)
0806 08:57:01.965475 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:01.965536 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:01.967450 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:01.968295 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:01.968307 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:01.969173 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:01.970041 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:01.970077 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.971677 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.971683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:01.973600 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:01.973614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.974440 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.974450 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:01.974683 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:01.975407 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:01.975429 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:01.978863 (+ 3434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:01.982957 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:01.982972 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:01.982997 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:01.983650 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:01.984065 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:01.984071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:01.984074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:01.984375 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:01.984942 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:01.984951 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:01.985320 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 08:57:01.985568 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 08:57:01.986021 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":346,"threads_started":1}
I20260806 08:57:01.988271 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:01.963954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:43715 (local address 127.22.98.62:36143)
0806 08:57:01.964499 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:01.964506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:01.965675 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 08:57:01.967789 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:01.967795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:01.967888 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:01.967992 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:01.970282 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.971460 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.973748 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.974192 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.974321 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:01.979251 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:01.979261 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:01.979266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:01.979317 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:01.982782 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:01.983780 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:01.983786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:01.983790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:01.983916 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:01.984504 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:01.984508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:01.984510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:01.984809 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:01.984949 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:01.987730 (+ 2781us) server_negotiation.cc:299] Negotiation successful
0806 08:57:01.987990 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":297,"threads_started":1}
I20260806 08:57:02.010064 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:01.995095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:45153 (local address 127.22.98.62:36143)
0806 08:57:01.995298 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:01.995305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:01.996314 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 08:57:01.996535 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:01.996541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:01.996633 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:01.996739 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:01.998228 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.999215 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.000917 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.001469 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.001593 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.004350 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:02.004355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:02.004358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:02.004392 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:02.007326 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.008066 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.008070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.008073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.008205 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.008570 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.008574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.008576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.008868 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:02.009013 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:02.009447 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 08:57:02.009692 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 08:57:02.010164 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:01.995082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.1:45153)
0806 08:57:01.996173 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:01.996197 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:01.996390 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:01.996925 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:01.996931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:01.997573 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:01.998073 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:01.998089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:01.999419 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:01.999425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.000766 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:02.000780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.001895 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.001902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.002049 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.002869 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:02.002898 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:02.004106 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:02.007471 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.007478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.007482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.007939 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.008316 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.008322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.008326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.008457 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.008987 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:02.008993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:02.009184 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 08:57:02.009305 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 08:57:02.009692 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":681,"threads_started":1}
I20260806 08:57:02.012678 24649 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:46179
I20260806 08:57:02.012681 24841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:46179 every 8 connection(s)
I20260806 08:57:02.013978 24649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:02.016063 24842 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36143
I20260806 08:57:02.016283 24842 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:02.016882 24842 heartbeater.cc:511] Master 127.22.98.62:36143 requested a full tablet report, sending...
I20260806 08:57:02.018817 24541 ts_manager.cc:194] Registered new tserver with Master: 0d516c77473d48158470cd8773c751af (127.22.98.1:46179)
I20260806 08:57:02.020325 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 24649
I20260806 08:57:02.020340 24541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:43715
I20260806 08:57:02.020527 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:02.021127 22920 external_mini_cluster.cc:1428] Setting key 26e4bfe0c8af6695d4182fbc9d59cae5
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:02.064791 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36143
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:02.413172 24849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:02.413512 24849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:02.413666 24849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:02.421236 24849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:02.421394 24849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:02.421526 24849 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:02.421651 24849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:02.432147 24849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:36143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:02.433637 24849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:02.435222 24849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:02.450575 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:02.450573 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:02.450623 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:02.451381 24849 server_base.cc:1034] running on GCE node
I20260806 08:57:02.452131 24849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:02.453176 24849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:02.454442 24849 hybrid_clock.cc:648] HybridClock initialized: now 1786006622454372 us; error 36 us; skew 500 ppm
I20260806 08:57:02.454828 24849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:02 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:02.458390 24849 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:02.460179 24849 webserver.cc:460] Webserver started at http://127.22.98.2:45579/ using document root <none> and password file <none>
I20260806 08:57:02.460928 24849 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:02.461043 24849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:02.461390 24849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:02.464681 24849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "e894d87dcc364de9806a694bb07cc293"
format_stamp: "Formatted at 2026-08-06 08:57:02 on dist-test-slave-jpj1"
server_key: "f8ac0cd4c1f967cecffb4ba0e27c4003"
server_key_iv: "9bae8f0f985799ba2693553fa6695a3b"
server_key_version: "encryptionkey@0"
I20260806 08:57:02.465575 24849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "e894d87dcc364de9806a694bb07cc293"
format_stamp: "Formatted at 2026-08-06 08:57:02 on dist-test-slave-jpj1"
server_key: "f8ac0cd4c1f967cecffb4ba0e27c4003"
server_key_iv: "9bae8f0f985799ba2693553fa6695a3b"
server_key_version: "encryptionkey@0"
I20260806 08:57:02.471740 24849 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 08:57:02.476259 24867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:02.477962 24849 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 08:57:02.478168 24849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "e894d87dcc364de9806a694bb07cc293"
format_stamp: "Formatted at 2026-08-06 08:57:02 on dist-test-slave-jpj1"
server_key: "f8ac0cd4c1f967cecffb4ba0e27c4003"
server_key_iv: "9bae8f0f985799ba2693553fa6695a3b"
server_key_version: "encryptionkey@0"
I20260806 08:57:02.478407 24849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:02.511526 24849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:02.516285 24849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:02.516536 24849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:02.519268 24849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:02.519367 24849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:02.519522 24849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:02.519618 24849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 06 08:57:02 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:02.546403 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:02.520111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:54205 (local address 127.22.98.62:36143)
0806 08:57:02.520702 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:02.520709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:02.521884 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 08:57:02.524761 (+ 2877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:02.524768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:02.524870 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:02.524994 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:02.528106 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.529092 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.532306 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.533049 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.533223 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.539213 (+ 5990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:02.539221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:02.539226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:02.539292 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:02.542343 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.543640 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.543649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.543653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.543826 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.544380 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.544384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.544387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.544714 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:02.544866 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:02.545734 (+ 868us) server_negotiation.cc:299] Negotiation successful
0806 08:57:02.546005 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":323,"threads_started":1}
I20260806 08:57:02.547957 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:02.520772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.2:54205)
0806 08:57:02.521673 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:02.521728 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:02.524367 (+ 2639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:02.525514 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:02.525532 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:02.526398 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:02.527739 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:02.527883 (+ 144us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.529325 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.529336 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.531948 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:02.531995 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.533341 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.533351 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.533619 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.534420 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:02.534452 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:02.538729 (+ 4277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:02.542551 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.542565 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.542590 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.543396 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.543968 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.543974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.543978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.544239 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.544858 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:02.544867 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:02.545304 (+ 437us) client_negotiation.cc:769] Sending connection context
0806 08:57:02.545583 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 08:57:02.546212 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":323,"threads_started":1}
I20260806 08:57:02.569943 24849 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:41721
I20260806 08:57:02.570024 25052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:41721 every 8 connection(s)
I20260806 08:57:02.571214 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:02.555199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:35041 (local address 127.22.98.62:36143)
0806 08:57:02.555411 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:02.555419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:02.556245 (+ 826us) server_negotiation.cc:407] Connection header received
0806 08:57:02.556464 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:02.556470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:02.556564 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:02.556677 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:02.558434 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.559343 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.561616 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.562039 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.562165 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.564726 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:02.564732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:02.564736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:02.564776 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:02.568267 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.569007 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.569013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.569017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.569183 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:02.569678 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:02.569684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:02.569687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:02.570178 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:02.570339 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:02.570776 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 08:57:02.570974 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 08:57:02.571460 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:02.555156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.2:35041)
0806 08:57:02.556099 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:02.556128 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:02.556317 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:02.556782 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:02.556789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:02.557578 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:02.558168 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:02.558189 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.559500 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.559506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.561462 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:02.561480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:02.562258 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:02.562265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:02.562442 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:02.563170 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:02.563197 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:02.564442 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:02.568419 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.568424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.568426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.568860 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.569320 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:02.569327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:02.569335 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:02.569532 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:02.570307 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:02.570312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:02.570523 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 08:57:02.570674 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 08:57:02.571040 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":581,"threads_started":1}
I20260806 08:57:02.571679 24849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:02.573746 25055 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36143
I20260806 08:57:02.574002 25055 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:02.574823 25055 heartbeater.cc:511] Master 127.22.98.62:36143 requested a full tablet report, sending...
I20260806 08:57:02.576265 24541 ts_manager.cc:194] Registered new tserver with Master: e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721)
I20260806 08:57:02.577093 24541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:54205
I20260806 08:57:02.577832 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 24849
I20260806 08:57:02.578032 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:02.578465 22920 external_mini_cluster.cc:1428] Setting key d28626feebd34de4e5d1618ac8566a29
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:02.626842 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36143
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:02.968134 25059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:02.968478 25059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:02.968634 25059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:02.976037 25059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:02.976169 25059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:02.976297 25059 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:02.976411 25059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:02.986503 25059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:36143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:02.987916 25059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:02.989378 25059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:03.003051 25068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:03.003147 25073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:03.003073 25071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:03.004060 25059 server_base.cc:1034] running on GCE node
I20260806 08:57:03.004765 25059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:03.005795 25059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:03.007061 25059 hybrid_clock.cc:648] HybridClock initialized: now 1786006623006967 us; error 70 us; skew 500 ppm
I20260806 08:57:03.007391 25059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:03 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:03.011252 25059 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:03.013051 25059 webserver.cc:460] Webserver started at http://127.22.98.3:33331/ using document root <none> and password file <none>
I20260806 08:57:03.013787 25059 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:03.013898 25059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:03.014215 25059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:03.017410 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "e95cb1e772044ba7a2fb908fc2c783e6"
format_stamp: "Formatted at 2026-08-06 08:57:03 on dist-test-slave-jpj1"
server_key: "b8858782653422478c11f33eaeeaa2fe"
server_key_iv: "188f4cf9c58ca1635f000b6c2987ae45"
server_key_version: "encryptionkey@0"
I20260806 08:57:03.018237 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "e95cb1e772044ba7a2fb908fc2c783e6"
format_stamp: "Formatted at 2026-08-06 08:57:03 on dist-test-slave-jpj1"
server_key: "b8858782653422478c11f33eaeeaa2fe"
server_key_iv: "188f4cf9c58ca1635f000b6c2987ae45"
server_key_version: "encryptionkey@0"
I20260806 08:57:03.023465 24842 heartbeater.cc:503] Master 127.22.98.62:36143 was elected leader, sending a full tablet report...
I20260806 08:57:03.023940 25059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 08:57:03.028358 25081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:03.029855 25059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:03.030025 25059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "e95cb1e772044ba7a2fb908fc2c783e6"
format_stamp: "Formatted at 2026-08-06 08:57:03 on dist-test-slave-jpj1"
server_key: "b8858782653422478c11f33eaeeaa2fe"
server_key_iv: "188f4cf9c58ca1635f000b6c2987ae45"
server_key_version: "encryptionkey@0"
I20260806 08:57:03.030236 25059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:03.057219 25059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:03.061648 25059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:03.061923 25059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:03.064762 25059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:03.064864 25059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:03.065019 25059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:03.065135 25059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 08:57:03 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:03.091308 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.065292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:58443 (local address 127.22.98.62:36143)
0806 08:57:03.065546 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.065554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.067127 (+ 1573us) server_negotiation.cc:407] Connection header received
0806 08:57:03.069505 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.069512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.069634 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.069719 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:03.072564 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.073581 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.076116 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.076727 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.076883 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.082810 (+ 5927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:03.082816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:03.082819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:03.082859 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:03.086470 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.087753 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.087761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.087765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.087954 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.088618 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.088634 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.088639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.089057 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:03.089202 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.090737 (+ 1535us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.090978 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 08:57:03.093191 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.066001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.3:58443)
0806 08:57:03.066921 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.066983 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.069145 (+ 2162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.070328 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.070337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.071247 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:03.072329 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.072366 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.073792 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.073796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.075932 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.075952 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.077065 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.077080 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.077498 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.078489 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:03.078533 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:03.082471 (+ 3938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:03.086702 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.086713 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.086742 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.087509 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.088113 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.088119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.088127 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.088456 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.089345 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:03.089353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:03.090098 (+ 745us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.090589 (+ 491us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.091364 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":355,"threads_started":1}
I20260806 08:57:03.114802 25059 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:36451
I20260806 08:57:03.114821 25263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:36451 every 8 connection(s)
I20260806 08:57:03.115084 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.100558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:46597 (local address 127.22.98.62:36143)
0806 08:57:03.100790 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.100797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.101470 (+ 673us) server_negotiation.cc:407] Connection header received
0806 08:57:03.101670 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.101675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.101744 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.101826 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:03.102999 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.103850 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.105782 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.106374 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.106499 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.109100 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:03.109106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:03.109109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:03.109146 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:03.111981 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.112914 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.112921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.112925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.113076 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.113445 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.113450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.113454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.113750 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:03.113910 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.114498 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.114727 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 08:57:03.115381 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.100709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.22.98.3:46597)
0806 08:57:03.101360 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.101380 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.101535 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.101872 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.101876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.102517 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:03.102844 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.102855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.104016 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.104021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.105614 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.105638 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.106564 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.106571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.106732 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.107441 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:03.107473 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:03.108874 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:03.112131 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.112137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.112141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.112778 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.113193 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.113199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.113203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.113340 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.113982 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:03.113991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:03.114226 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.114408 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.114795 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":360,"threads_started":1}
I20260806 08:57:03.116837 25059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:03.118644 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 25059
I20260806 08:57:03.118839 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:03.119268 22920 external_mini_cluster.cc:1428] Setting key 92afada84f1e086da63bd91484c088d4
I20260806 08:57:03.120719 25264 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36143
I20260806 08:57:03.120959 25264 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:03.121707 25264 heartbeater.cc:511] Master 127.22.98.62:36143 requested a full tablet report, sending...
I20260806 08:57:03.123124 24538 ts_manager.cc:194] Registered new tserver with Master: e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451)
I20260806 08:57:03.124091 24538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:58443
I20260806 08:57:03.135160 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:57:03 dist-test-slave-jpj1 krb5kdc[24478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 08:57:03 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:03.186698 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.172794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.22.98.62:36143)
0806 08:57:03.173043 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.173051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.173261 (+ 210us) server_negotiation.cc:407] Connection header received
0806 08:57:03.173459 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.173463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.173534 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.173634 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:03.175038 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.175864 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.177125 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.177521 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.177642 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.181486 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:03.181491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:03.181495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:03.181533 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:03.184164 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.184798 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.184803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.184805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.184930 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.185302 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.185307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.185309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.185598 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:03.185865 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.186135 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.186356 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 08:57:03.191442 24538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:03.198701 24538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:03.225678 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.211900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:41721 (local address 127.0.0.1:41872)
0806 08:57:03.214927 (+ 3027us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.214966 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.215162 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.216339 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.216349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.216415 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.216882 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.216896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.219158 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.219165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.221908 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.221927 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.224386 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.224394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.224696 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.224760 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.224916 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.225144 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2743,"spinlock_wait_cycles":9472,"thread_start_us":2550,"threads_started":1}
I20260806 08:57:03.225755 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.212035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.22.98.2:41721)
0806 08:57:03.214429 (+ 2394us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.214439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.215360 (+ 921us) server_negotiation.cc:407] Connection header received
0806 08:57:03.215476 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.215483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.215814 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.216032 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.217092 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.218948 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.222152 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.223742 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.223876 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.224149 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.225247 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.225440 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2214,"thread_start_us":2064,"threads_started":1}
I20260806 08:57:03.228531 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.212472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.22.98.1:46179)
0806 08:57:03.213953 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.213965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.216556 (+ 2591us) server_negotiation.cc:407] Connection header received
0806 08:57:03.216689 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.216699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.216952 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.217168 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.218089 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.221204 (+ 3115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.225973 (+ 4769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.227458 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.227622 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.227848 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.228080 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.228254 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1287,"thread_start_us":1126,"threads_started":1}
I20260806 08:57:03.229179 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.211999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:46179 (local address 127.0.0.1:38026)
0806 08:57:03.216236 (+ 4237us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.216256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.216404 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.217398 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.217404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.217433 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.217854 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.217866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.221430 (+ 3564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.221435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.225760 (+ 4325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.225776 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.227697 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.227704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.227890 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.227923 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.228006 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.228151 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2793,"spinlock_wait_cycles":2304,"thread_start_us":2642,"threads_started":1}
I20260806 08:57:03.230558 24943 tablet_service.cc:1467] Processing CreateTablet for tablet b2e083b9df6d4b26909911292ca30cdf (DEFAULT_TABLE table=test-table [id=6e47a316dc10420199f1e2127e398a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.232127 24943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e083b9df6d4b26909911292ca30cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.232870 24735 tablet_service.cc:1467] Processing CreateTablet for tablet b2e083b9df6d4b26909911292ca30cdf (DEFAULT_TABLE table=test-table [id=6e47a316dc10420199f1e2127e398a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.234517 24735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e083b9df6d4b26909911292ca30cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.236007 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.215749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.22.98.3:36451)
0806 08:57:03.226541 (+ 10792us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.226551 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.226587 (+ 36us) server_negotiation.cc:407] Connection header received
0806 08:57:03.226693 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.226703 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.226986 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.227211 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.227926 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.229514 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.231832 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.233459 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.233597 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.233848 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.235371 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.235689 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10596,"thread_start_us":10379,"threads_started":1}
I20260806 08:57:03.236301 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.215387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:36451 (local address 127.0.0.1:50132)
0806 08:57:03.224924 (+ 9537us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.224942 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.225054 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.227203 (+ 2149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.227208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.227238 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.227686 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.227700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.229713 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.229718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.231654 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.231671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.234005 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.234012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.234237 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.234274 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.234730 (+ 456us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.234885 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9398,"thread_start_us":9238,"threads_started":1}
I20260806 08:57:03.243840 25149 tablet_service.cc:1467] Processing CreateTablet for tablet b2e083b9df6d4b26909911292ca30cdf (DEFAULT_TABLE table=test-table [id=6e47a316dc10420199f1e2127e398a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.245687 25149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e083b9df6d4b26909911292ca30cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.245873 25300 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Bootstrap starting.
I20260806 08:57:03.247674 25301 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Bootstrap starting.
I20260806 08:57:03.249521 25300 tablet_bootstrap.cc:654] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.250844 25300 log.cc:824] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:03.251451 25301 tablet_bootstrap.cc:654] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.252833 25301 log.cc:824] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:03.256165 25300 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: No bootstrap required, opened a new log
I20260806 08:57:03.256172 25301 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: No bootstrap required, opened a new log
I20260806 08:57:03.256484 25301 ts_tablet_manager.cc:1397] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260806 08:57:03.256484 25300 ts_tablet_manager.cc:1397] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260806 08:57:03.259924 25305 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Bootstrap starting.
I20260806 08:57:03.264322 25305 tablet_bootstrap.cc:654] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.265564 25305 log.cc:824] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:03.267828 25301 raft_consensus.cc:348] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.268348 25301 raft_consensus.cc:374] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.268460 25301 raft_consensus.cc:729] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d516c77473d48158470cd8773c751af, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.269209 25301 consensus_queue.cc:260] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.269119 25300 raft_consensus.cc:348] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.269611 25300 raft_consensus.cc:374] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.269871 25300 raft_consensus.cc:729] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e894d87dcc364de9806a694bb07cc293, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.270537 25300 consensus_queue.cc:260] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.271265 25301 ts_tablet_manager.cc:1428] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260806 08:57:03.272248 25055 heartbeater.cc:503] Master 127.22.98.62:36143 was elected leader, sending a full tablet report...
I20260806 08:57:03.272825 25305 tablet_bootstrap.cc:492] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: No bootstrap required, opened a new log
I20260806 08:57:03.273141 25300 ts_tablet_manager.cc:1428] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Time spent starting tablet: real 0.017s user 0.008s sys 0.006s
I20260806 08:57:03.273155 25305 ts_tablet_manager.cc:1397] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260806 08:57:03.282130 25305 raft_consensus.cc:348] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.282585 25305 raft_consensus.cc:374] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.282693 25305 raft_consensus.cc:729] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95cb1e772044ba7a2fb908fc2c783e6, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.283413 25305 consensus_queue.cc:260] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.284679 25264 heartbeater.cc:503] Master 127.22.98.62:36143 was elected leader, sending a full tablet report...
I20260806 08:57:03.285236 25305 ts_tablet_manager.cc:1428] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
W20260806 08:57:03.323829 25056 tablet.cc:2369] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:57:03.370585 25268 tablet.cc:2369] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:03.449151 25311 raft_consensus.cc:482] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:03.449514 25311 raft_consensus.cc:504] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.451754 25311 leader_election.cc:288] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721), 0d516c77473d48158470cd8773c751af (127.22.98.1:46179)
I20260806 08:57:03.458415 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.452120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:48263 (local address 127.22.98.2:41721)
0806 08:57:03.452343 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.452350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.452370 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:57:03.452597 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.452605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.452701 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.452799 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.453429 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.454403 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.456177 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.457326 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.457431 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.457513 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.458087 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.458194 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 08:57:03.458482 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.451879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:41721 (local address 127.22.98.3:48263)
0806 08:57:03.452285 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.452307 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.452435 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.452919 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.452925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.452961 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.453267 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.453281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.454540 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.454544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.456013 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.456026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.457509 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.457516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.457861 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.457899 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.458030 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.458174 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260806 08:57:03.459510 24972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e083b9df6d4b26909911292ca30cdf" candidate_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293" is_pre_election: true
I20260806 08:57:03.460055 24972 raft_consensus.cc:2454] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e95cb1e772044ba7a2fb908fc2c783e6 in term 0.
I20260806 08:57:03.460788 25083 leader_election.cc:302] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e894d87dcc364de9806a694bb07cc293, e95cb1e772044ba7a2fb908fc2c783e6; no voters:
I20260806 08:57:03.461273 25311 raft_consensus.cc:2781] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:03.461390 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.452567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:45735 (local address 127.22.98.1:46179)
0806 08:57:03.452792 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.452799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.453466 (+ 667us) server_negotiation.cc:407] Connection header received
0806 08:57:03.453689 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.453693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.453772 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.453850 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.455710 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.456809 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.459143 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.460250 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.460376 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.460475 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.460951 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.461097 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 08:57:03.461493 25311 raft_consensus.cc:482] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:03.461544 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.452101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:46179 (local address 127.22.98.3:45735)
0806 08:57:03.453359 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.453379 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.453503 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.453892 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.453896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.453922 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.454384 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.454396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.456977 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.456982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.458976 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.458989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.460480 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.460486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.460755 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.460796 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.460896 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.461026 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"thread_start_us":828,"threads_started":1}
I20260806 08:57:03.461611 25311 raft_consensus.cc:3037] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.462757 24766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e083b9df6d4b26909911292ca30cdf" candidate_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d516c77473d48158470cd8773c751af" is_pre_election: true
I20260806 08:57:03.463330 24766 raft_consensus.cc:2454] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e95cb1e772044ba7a2fb908fc2c783e6 in term 0.
I20260806 08:57:03.464459 25311 raft_consensus.cc:504] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.465238 25311 leader_election.cc:288] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [CANDIDATE]: Term 1 election: Requested vote from peers e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721), 0d516c77473d48158470cd8773c751af (127.22.98.1:46179)
I20260806 08:57:03.465606 24972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e083b9df6d4b26909911292ca30cdf" candidate_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293"
I20260806 08:57:03.465724 24766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e083b9df6d4b26909911292ca30cdf" candidate_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d516c77473d48158470cd8773c751af"
I20260806 08:57:03.465842 24972 raft_consensus.cc:3037] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.465966 24766 raft_consensus.cc:3037] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.469259 24972 raft_consensus.cc:2454] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e95cb1e772044ba7a2fb908fc2c783e6 in term 1.
I20260806 08:57:03.469331 24766 raft_consensus.cc:2454] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e95cb1e772044ba7a2fb908fc2c783e6 in term 1.
I20260806 08:57:03.469698 25083 leader_election.cc:302] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e894d87dcc364de9806a694bb07cc293, e95cb1e772044ba7a2fb908fc2c783e6; no voters:
I20260806 08:57:03.470006 25311 raft_consensus.cc:2781] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:03.470638 25311 raft_consensus.cc:686] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 LEADER]: Becoming Leader. State: Replica: e95cb1e772044ba7a2fb908fc2c783e6, State: Running, Role: LEADER
I20260806 08:57:03.471174 25311 consensus_queue.cc:237] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.476315 24541 catalog_manager.cc:5495] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } health_report { overall_health: HEALTHY } } }
W20260806 08:57:03.516722 24843 tablet.cc:2369] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:03.534463 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.528638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.22.98.3:36451)
0806 08:57:03.528851 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.528856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.528879 (+ 23us) server_negotiation.cc:407] Connection header received
0806 08:57:03.529122 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.529127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.529218 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.529349 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:03.529731 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.530459 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.532012 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.532560 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.532672 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.533226 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:03.533847 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:03.533982 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.534119 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.534204 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 08:57:03.544404 24972 raft_consensus.cc:1261] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Refusing update from remote peer e95cb1e772044ba7a2fb908fc2c783e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:03.545804 24766 raft_consensus.cc:1261] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Refusing update from remote peer e95cb1e772044ba7a2fb908fc2c783e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:03.545545 25315 consensus_queue.cc:1035] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:03.546650 25315 consensus_queue.cc:1035] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:03.563113 25324 mvcc.cc:205] Tried to move back new op lower bound from 7315483130024386560 to 7315483129742454784. Current Snapshot: MvccSnapshot[applied={T|T < 7315483130024386560}]
I20260806 08:57:03.589695 24538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:03.590109 24538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:03.594316 24538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Sending DeleteTablet for 3 replicas of tablet b2e083b9df6d4b26909911292ca30cdf
I20260806 08:57:03.595892 24943 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e083b9df6d4b26909911292ca30cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:41872
I20260806 08:57:03.596309 24735 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e083b9df6d4b26909911292ca30cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:38026
I20260806 08:57:03.597052 25149 tablet_service.cc:1514] Processing DeleteTablet for tablet b2e083b9df6d4b26909911292ca30cdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:50132
I20260806 08:57:03.597492 24538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:03.598594 24538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:03.600373 25340 tablet_replica.cc:330] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: stopping tablet replica
I20260806 08:57:03.601008 25340 raft_consensus.cc:2229] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:03.601866 25340 raft_consensus.cc:2258] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:03.602860 25338 tablet_replica.cc:330] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: stopping tablet replica
I20260806 08:57:03.603358 25338 raft_consensus.cc:2229] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:03.605752 25338 raft_consensus.cc:2258] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:03.607599 24942 tablet_service.cc:1467] Processing CreateTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 (DEFAULT_TABLE table=test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.608083 24734 tablet_service.cc:1467] Processing CreateTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 (DEFAULT_TABLE table=test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.608237 24942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d172a3e314a4529bf11b176c4cc47b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.608559 24734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d172a3e314a4529bf11b176c4cc47b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.612231 25149 tablet_service.cc:1467] Processing CreateTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 (DEFAULT_TABLE table=test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:03.612752 25149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d172a3e314a4529bf11b176c4cc47b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.617201 25300 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Bootstrap starting.
I20260806 08:57:03.619141 25300 tablet_bootstrap.cc:654] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.622902 25300 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: No bootstrap required, opened a new log
I20260806 08:57:03.623135 25300 ts_tablet_manager.cc:1397] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 08:57:03.624055 25300 raft_consensus.cc:348] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.624375 25300 raft_consensus.cc:374] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.624539 25300 raft_consensus.cc:729] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e894d87dcc364de9806a694bb07cc293, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.624949 25300 consensus_queue.cc:260] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.625485 25338 ts_tablet_manager.cc:1905] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:03.625985 25300 ts_tablet_manager.cc:1428] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:57:03.626212 25301 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Bootstrap starting.
I20260806 08:57:03.628063 25301 tablet_bootstrap.cc:654] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.628540 25340 ts_tablet_manager.cc:1905] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:03.631858 25305 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Bootstrap starting.
I20260806 08:57:03.633883 25305 tablet_bootstrap.cc:654] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.636385 25338 ts_tablet_manager.cc:1918] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:03.637951 25338 log.cc:1196] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/b2e083b9df6d4b26909911292ca30cdf
I20260806 08:57:03.638631 25338 ts_tablet_manager.cc:1939] T b2e083b9df6d4b26909911292ca30cdf P 0d516c77473d48158470cd8773c751af: Deleting consensus metadata
I20260806 08:57:03.640799 24521 catalog_manager.cc:4841] TS 0d516c77473d48158470cd8773c751af (127.22.98.1:46179): tablet b2e083b9df6d4b26909911292ca30cdf (table test-table [id=6e47a316dc10420199f1e2127e398a85]) successfully deleted
I20260806 08:57:03.642077 25337 tablet_replica.cc:330] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: stopping tablet replica
I20260806 08:57:03.642344 25305 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: No bootstrap required, opened a new log
I20260806 08:57:03.642519 25305 ts_tablet_manager.cc:1397] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 08:57:03.642494 25337 raft_consensus.cc:2229] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:03.642999 25337 raft_consensus.cc:2258] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:03.643460 25305 raft_consensus.cc:348] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.643775 25305 raft_consensus.cc:374] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.643874 25305 raft_consensus.cc:729] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95cb1e772044ba7a2fb908fc2c783e6, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.644284 25305 consensus_queue.cc:260] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.645078 25305 ts_tablet_manager.cc:1428] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 08:57:03.647876 25340 ts_tablet_manager.cc:1918] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:03.648099 25340 log.cc:1196] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/b2e083b9df6d4b26909911292ca30cdf
I20260806 08:57:03.648749 25340 ts_tablet_manager.cc:1939] T b2e083b9df6d4b26909911292ca30cdf P e95cb1e772044ba7a2fb908fc2c783e6: Deleting consensus metadata
I20260806 08:57:03.649842 25337 ts_tablet_manager.cc:1905] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:03.650514 25301 tablet_bootstrap.cc:492] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: No bootstrap required, opened a new log
I20260806 08:57:03.650695 25301 ts_tablet_manager.cc:1397] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.007s
I20260806 08:57:03.651607 24521 catalog_manager.cc:4841] TS e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451): tablet b2e083b9df6d4b26909911292ca30cdf (table test-table [id=6e47a316dc10420199f1e2127e398a85]) successfully deleted
I20260806 08:57:03.651726 25301 raft_consensus.cc:348] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.651945 25301 raft_consensus.cc:374] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.652041 25301 raft_consensus.cc:729] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d516c77473d48158470cd8773c751af, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.652356 25301 consensus_queue.cc:260] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.652628 25308 raft_consensus.cc:482] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:03.652856 25308 raft_consensus.cc:504] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.653417 25301 ts_tablet_manager.cc:1428] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:03.654973 25308 leader_election.cc:288] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721), e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451)
I20260806 08:57:03.658342 25337 ts_tablet_manager.cc:1918] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:03.658560 25337 log.cc:1196] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/b2e083b9df6d4b26909911292ca30cdf
I20260806 08:57:03.659198 25337 ts_tablet_manager.cc:1939] T b2e083b9df6d4b26909911292ca30cdf P e894d87dcc364de9806a694bb07cc293: Deleting consensus metadata
I20260806 08:57:03.661772 24518 catalog_manager.cc:4841] TS e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721): tablet b2e083b9df6d4b26909911292ca30cdf (table test-table [id=6e47a316dc10420199f1e2127e398a85]) successfully deleted
I20260806 08:57:03.672576 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.656506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54477 (local address 127.22.98.3:36451)
0806 08:57:03.656888 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.656895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.664395 (+ 7500us) server_negotiation.cc:407] Connection header received
0806 08:57:03.664711 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.664717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.664799 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.664900 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.665782 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.666960 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.669925 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.671289 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.671429 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.671560 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.672104 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.672262 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260806 08:57:03.672655 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.655739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:36451 (local address 127.22.98.1:54477)
0806 08:57:03.664241 (+ 8502us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.664267 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.664536 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.665144 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.665150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.665180 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.665570 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.665583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.667109 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.667115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.669709 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.669726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.671744 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.671751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.671917 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.671949 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.672052 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.672226 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8310,"mutex_wait_us":53,"spinlock_wait_cycles":18688,"thread_start_us":9170,"threads_started":1}
I20260806 08:57:03.672853 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.656687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:39695 (local address 127.22.98.2:41721)
0806 08:57:03.657030 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.657037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.665318 (+ 8281us) server_negotiation.cc:407] Connection header received
0806 08:57:03.665611 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.665616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.665694 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.665756 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:03.666657 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.667783 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.670465 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.671695 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.671796 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.671890 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.672442 (+ 552us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.672577 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260806 08:57:03.673475 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.656108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:41721 (local address 127.22.98.1:39695)
0806 08:57:03.665216 (+ 9108us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.665233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.665434 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.665975 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.665980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.666003 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:03.666434 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.666444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.667955 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.667960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.670265 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.670281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.672083 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.672089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.672254 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.672284 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.673032 (+ 748us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.673162 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8960,"thread_start_us":7738,"threads_started":1}
I20260806 08:57:03.673892 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d172a3e314a4529bf11b176c4cc47b0" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" is_pre_election: true
I20260806 08:57:03.674340 25187 raft_consensus.cc:2454] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 0.
I20260806 08:57:03.674916 24971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d172a3e314a4529bf11b176c4cc47b0" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293" is_pre_election: true
I20260806 08:57:03.675310 24971 raft_consensus.cc:2454] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 0.
I20260806 08:57:03.675230 24673 leader_election.cc:302] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d516c77473d48158470cd8773c751af, e95cb1e772044ba7a2fb908fc2c783e6; no voters:
I20260806 08:57:03.675700 25308 raft_consensus.cc:2781] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:03.675863 25308 raft_consensus.cc:482] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:03.675974 25308 raft_consensus.cc:3037] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.678308 25308 raft_consensus.cc:504] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.678999 25308 leader_election.cc:288] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 election: Requested vote from peers e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721), e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451)
I20260806 08:57:03.679471 24971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d172a3e314a4529bf11b176c4cc47b0" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293"
I20260806 08:57:03.679733 24971 raft_consensus.cc:3037] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.679759 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d172a3e314a4529bf11b176c4cc47b0" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e95cb1e772044ba7a2fb908fc2c783e6"
I20260806 08:57:03.679994 25187 raft_consensus.cc:3037] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:03.682207 25187 raft_consensus.cc:2454] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 1.
I20260806 08:57:03.682626 24673 leader_election.cc:302] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d516c77473d48158470cd8773c751af, e95cb1e772044ba7a2fb908fc2c783e6; no voters:
I20260806 08:57:03.682940 25308 raft_consensus.cc:2781] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:03.683044 24971 raft_consensus.cc:2454] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 1.
I20260806 08:57:03.683147 25308 raft_consensus.cc:686] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 1 LEADER]: Becoming Leader. State: Replica: 0d516c77473d48158470cd8773c751af, State: Running, Role: LEADER
I20260806 08:57:03.683631 25308 consensus_queue.cc:237] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } }
I20260806 08:57:03.687719 24538 catalog_manager.cc:5495] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d516c77473d48158470cd8773c751af (127.22.98.1). New cstate: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } health_report { overall_health: UNKNOWN } } }
Aug 06 08:57:03 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:03.741777 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.726314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36143 (local address 127.0.0.1:59346)
0806 08:57:03.727033 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:03.727049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:03.727201 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:03.727512 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:03.727516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:03.728344 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:03.728612 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.728622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.730025 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.730030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.731164 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:03.731179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.731892 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.731898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:03.732098 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.732799 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:03.732817 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:03.735928 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:03.739172 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.739185 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.739194 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.739660 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.740004 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:03.740009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:03.740013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:03.740176 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:03.740705 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:03.740714 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:03.740928 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 08:57:03.741062 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 08:57:03.741351 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":486,"threads_started":1}
I20260806 08:57:03.741855 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:03.726214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.22.98.62:36143)
0806 08:57:03.726708 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:03.726715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:03.727139 (+ 424us) server_negotiation.cc:407] Connection header received
0806 08:57:03.727313 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:03.727318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:03.727384 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:03.727471 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:03.728773 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.729891 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.731295 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:03.731735 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:03.731868 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:03.736161 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:03.736169 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:03.736175 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:03.736216 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:03.739028 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.739772 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.739778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.739781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.739906 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:03.740282 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:03.740288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:03.740291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:03.740572 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:03.740714 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:03.741187 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 08:57:03.741476 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":352,"thread_start_us":242,"threads_started":1}
I20260806 08:57:03.748564 24538 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:59346:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 08:57:03.750025 24538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:03.757972 25149 tablet_service.cc:1467] Processing CreateTablet for tablet c4debc13e05c4051a9324a4d540e54c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60cb5cbc782b4663bf69ffa973c24b19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:03.757931 24734 tablet_service.cc:1467] Processing CreateTablet for tablet c4debc13e05c4051a9324a4d540e54c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60cb5cbc782b4663bf69ffa973c24b19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:03.758546 24734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4debc13e05c4051a9324a4d540e54c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.758587 25149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4debc13e05c4051a9324a4d540e54c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.764706 24943 tablet_service.cc:1467] Processing CreateTablet for tablet c4debc13e05c4051a9324a4d540e54c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60cb5cbc782b4663bf69ffa973c24b19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:03.765266 24943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4debc13e05c4051a9324a4d540e54c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:03.766261 25301 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: Bootstrap starting.
I20260806 08:57:03.766328 25305 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: Bootstrap starting.
I20260806 08:57:03.768116 25301 tablet_bootstrap.cc:654] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.768208 25305 tablet_bootstrap.cc:654] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.772682 25300 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: Bootstrap starting.
I20260806 08:57:03.774451 25300 tablet_bootstrap.cc:654] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:03.780285 25300 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: No bootstrap required, opened a new log
I20260806 08:57:03.780279 25305 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: No bootstrap required, opened a new log
I20260806 08:57:03.780493 25305 ts_tablet_manager.cc:1397] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260806 08:57:03.780616 25300 ts_tablet_manager.cc:1397] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 08:57:03.780874 25301 tablet_bootstrap.cc:492] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: No bootstrap required, opened a new log
I20260806 08:57:03.781070 25301 ts_tablet_manager.cc:1397] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260806 08:57:03.781508 25305 raft_consensus.cc:348] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.781558 25300 raft_consensus.cc:348] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.781806 25300 raft_consensus.cc:374] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.781883 25300 raft_consensus.cc:729] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e894d87dcc364de9806a694bb07cc293, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.781922 25301 raft_consensus.cc:348] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.782127 25305 raft_consensus.cc:374] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.782157 25301 raft_consensus.cc:374] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:03.782227 25305 raft_consensus.cc:729] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95cb1e772044ba7a2fb908fc2c783e6, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.782253 25301 raft_consensus.cc:729] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d516c77473d48158470cd8773c751af, State: Initialized, Role: FOLLOWER
I20260806 08:57:03.782259 25300 consensus_queue.cc:260] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.782526 25301 consensus_queue.cc:260] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.782632 25305 consensus_queue.cc:260] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:03.782945 25308 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:03.783236 25308 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:03.782863 25310 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:03.783447 25305 ts_tablet_manager.cc:1428] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 08:57:03.783556 25310 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:03.783882 25301 ts_tablet_manager.cc:1428] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 08:57:03.784799 25333 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:03.785168 25333 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:03.785749 25300 ts_tablet_manager.cc:1428] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 08:57:04.112504 25308 raft_consensus.cc:482] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:04.112736 25308 raft_consensus.cc:504] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:04.113360 25308 leader_election.cc:288] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451), e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721)
I20260806 08:57:04.114007 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4debc13e05c4051a9324a4d540e54c5" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" is_pre_election: true
I20260806 08:57:04.114060 24971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4debc13e05c4051a9324a4d540e54c5" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293" is_pre_election: true
I20260806 08:57:04.114296 25187 raft_consensus.cc:2454] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 0.
I20260806 08:57:04.114377 24971 raft_consensus.cc:2454] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 0.
I20260806 08:57:04.114827 24671 leader_election.cc:302] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d516c77473d48158470cd8773c751af, e894d87dcc364de9806a694bb07cc293; no voters:
I20260806 08:57:04.115288 25308 raft_consensus.cc:2781] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:04.115491 25308 raft_consensus.cc:482] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:04.115644 25308 raft_consensus.cc:3037] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:04.117683 25308 raft_consensus.cc:504] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:04.118268 25308 leader_election.cc:288] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 election: Requested vote from peers e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451), e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721)
I20260806 08:57:04.118796 25187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4debc13e05c4051a9324a4d540e54c5" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e95cb1e772044ba7a2fb908fc2c783e6"
I20260806 08:57:04.118902 24971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4debc13e05c4051a9324a4d540e54c5" candidate_uuid: "0d516c77473d48158470cd8773c751af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e894d87dcc364de9806a694bb07cc293"
I20260806 08:57:04.119055 25187 raft_consensus.cc:3037] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:04.119132 24971 raft_consensus.cc:3037] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:04.122121 24971 raft_consensus.cc:2454] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 1.
I20260806 08:57:04.122128 25187 raft_consensus.cc:2454] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d516c77473d48158470cd8773c751af in term 1.
I20260806 08:57:04.122645 24673 leader_election.cc:302] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d516c77473d48158470cd8773c751af, e95cb1e772044ba7a2fb908fc2c783e6; no voters:
I20260806 08:57:04.122947 25308 raft_consensus.cc:2781] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:04.123186 25308 raft_consensus.cc:686] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [term 1 LEADER]: Becoming Leader. State: Replica: 0d516c77473d48158470cd8773c751af, State: Running, Role: LEADER
I20260806 08:57:04.123512 25308 consensus_queue.cc:237] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } }
I20260806 08:57:04.125236 25344 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: TxnStatusTablet state changed. Reason: New leader 0d516c77473d48158470cd8773c751af. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.125576 25344 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:04.126138 25369 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 08:57:04.126417 25369 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 08:57:04.127249 24541 catalog_manager.cc:5495] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d516c77473d48158470cd8773c751af (127.22.98.1). New cstate: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } health_report { overall_health: UNKNOWN } } }
I20260806 08:57:04.226691 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.220788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.22.98.1:46179)
0806 08:57:04.221384 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.221389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.221404 (+ 15us) server_negotiation.cc:407] Connection header received
0806 08:57:04.221489 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.221493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.221576 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.221736 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:04.222116 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.223028 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.224352 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.224801 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.224912 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.225368 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:04.226001 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:04.226143 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.226277 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.226362 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":294,"threads_started":1}
I20260806 08:57:04.226791 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.220690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:46179 (local address 127.0.0.1:38040)
0806 08:57:04.220951 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:04.220974 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:04.221129 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:04.221780 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:04.221785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:04.221798 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:04.221982 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:04.221991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.223158 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.223163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:04.224200 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:04.224222 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.224950 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.224956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:04.225108 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.225171 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:04.226111 (+ 940us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:04.226135 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 08:57:04.226227 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 08:57:04.226385 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 08:57:04.232237 24971 raft_consensus.cc:1261] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Refusing update from remote peer 0d516c77473d48158470cd8773c751af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:04.232580 25187 raft_consensus.cc:1261] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Refusing update from remote peer 0d516c77473d48158470cd8773c751af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:04.232967 25344 consensus_queue.cc:1035] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:04.233448 25308 consensus_queue.cc:1035] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:04.244354 25310 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: TxnStatusTablet state changed. Reason: New leader 0d516c77473d48158470cd8773c751af. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.244715 25310 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:04.246775 25333 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: TxnStatusTablet state changed. Reason: New leader 0d516c77473d48158470cd8773c751af. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.247046 25333 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:04.247961 25377 mvcc.cc:205] Tried to move back new op lower bound from 7315483132847849472 to 7315483132412440576. Current Snapshot: MvccSnapshot[applied={T|T < 7315483132847849472}]
I20260806 08:57:04.252245 25308 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.252731 25308 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:04.253388 25344 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.253654 25344 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P 0d516c77473d48158470cd8773c751af: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:04.254318 25381 mvcc.cc:205] Tried to move back new op lower bound from 7315483132847849472 to 7315483132412440576. Current Snapshot: MvccSnapshot[applied={T|T < 7315483132847849472}]
I20260806 08:57:04.257539 25310 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.257838 25310 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e894d87dcc364de9806a694bb07cc293: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:04.253000 25308 consensus_queue.cc:1035] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260806 08:57:04.261516 25333 tablet_replica.cc:439] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d516c77473d48158470cd8773c751af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d516c77473d48158470cd8773c751af" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 46179 } } peers { permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 } } peers { permanent_uuid: "e894d87dcc364de9806a694bb07cc293" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 41721 } } }
I20260806 08:57:04.261830 25333 tablet_replica.cc:442] T c4debc13e05c4051a9324a4d540e54c5 P e95cb1e772044ba7a2fb908fc2c783e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:04.268450 25344 consensus_queue.cc:1035] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e95cb1e772044ba7a2fb908fc2c783e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 36451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:04.271688 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.261036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.22.98.1:46179)
0806 08:57:04.264743 (+ 3707us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.264749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.264767 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:57:04.264832 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.264838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.264923 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.265557 (+ 634us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:04.265738 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.266790 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.269030 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.269663 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.270285 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.270394 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:04.270515 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:04.271204 (+ 689us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.271259 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.271353 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3589}
I20260806 08:57:04.299355 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.293646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:58107 (local address 127.22.98.1:46179)
0806 08:57:04.293841 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.293846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.294432 (+ 586us) server_negotiation.cc:407] Connection header received
0806 08:57:04.294656 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.294660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.294717 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.294803 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:04.295348 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.296199 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.297603 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.298018 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.298137 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.298549 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:04.298657 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:04.298736 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.298859 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.298937 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260806 08:57:04.299382 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.293666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:46179 (local address 127.22.98.1:58107)
0806 08:57:04.294336 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:04.294352 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:04.294503 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:04.294840 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:04.294845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:04.294856 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:04.295211 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:04.295220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.296340 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.296346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:04.297423 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:04.297436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.298168 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.298175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:04.298323 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.298384 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:04.298752 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:04.298764 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 08:57:04.298834 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 08:57:04.298971 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":349,"threads_started":1}
I20260806 08:57:04.367767 24541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:04.368024 24541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:04.371124 24541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9463c8114a49403dbc3826cb36e6ca88: Sending DeleteTablet for 3 replicas of tablet 7d172a3e314a4529bf11b176c4cc47b0
I20260806 08:57:04.372032 24735 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:04 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:38026
I20260806 08:57:04.372247 24943 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:04 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:41872
I20260806 08:57:04.372742 25149 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d172a3e314a4529bf11b176c4cc47b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:04 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:50132
I20260806 08:57:04.383117 25403 tablet_replica.cc:330] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: stopping tablet replica
I20260806 08:57:04.383370 25403 raft_consensus.cc:2229] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:04.384066 25403 raft_consensus.cc:2258] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:04.384529 25404 tablet_replica.cc:330] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: stopping tablet replica
I20260806 08:57:04.384795 25404 raft_consensus.cc:2229] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:04.385079 25404 raft_consensus.cc:2258] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:04.386209 25404 ts_tablet_manager.cc:1905] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:04.386948 25403 ts_tablet_manager.cc:1905] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:04.388815 25406 tablet_replica.cc:330] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: stopping tablet replica
I20260806 08:57:04.389012 25406 raft_consensus.cc:2229] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:04.389309 25406 raft_consensus.cc:2258] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:04.390865 25406 ts_tablet_manager.cc:1905] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:04.393743 25404 ts_tablet_manager.cc:1918] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:04.393934 25404 log.cc:1196] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/7d172a3e314a4529bf11b176c4cc47b0
I20260806 08:57:04.394488 25404 ts_tablet_manager.cc:1939] T 7d172a3e314a4529bf11b176c4cc47b0 P e894d87dcc364de9806a694bb07cc293: Deleting consensus metadata
I20260806 08:57:04.395747 24518 catalog_manager.cc:4841] TS e894d87dcc364de9806a694bb07cc293 (127.22.98.2:41721): tablet 7d172a3e314a4529bf11b176c4cc47b0 (table test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]) successfully deleted
I20260806 08:57:04.398062 25406 ts_tablet_manager.cc:1918] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:04.398188 25406 log.cc:1196] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/7d172a3e314a4529bf11b176c4cc47b0
I20260806 08:57:04.398664 25406 ts_tablet_manager.cc:1939] T 7d172a3e314a4529bf11b176c4cc47b0 P e95cb1e772044ba7a2fb908fc2c783e6: Deleting consensus metadata
I20260806 08:57:04.403892 25403 ts_tablet_manager.cc:1918] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:04.404134 25403 log.cc:1196] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/7d172a3e314a4529bf11b176c4cc47b0
I20260806 08:57:04.404682 25403 ts_tablet_manager.cc:1939] T 7d172a3e314a4529bf11b176c4cc47b0 P 0d516c77473d48158470cd8773c751af: Deleting consensus metadata
I20260806 08:57:04.405975 24521 catalog_manager.cc:4841] TS 0d516c77473d48158470cd8773c751af (127.22.98.1:46179): tablet 7d172a3e314a4529bf11b176c4cc47b0 (table test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]) successfully deleted
I20260806 08:57:04.406852 24521 catalog_manager.cc:4841] TS e95cb1e772044ba7a2fb908fc2c783e6 (127.22.98.3:36451): tablet 7d172a3e314a4529bf11b176c4cc47b0 (table test-table [id=ccf3a3fb783a4a97b31f0d7c769594fc]) successfully deleted
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:04.425582 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.399983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.22.98.1:46179)
0806 08:57:04.400162 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.400168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.406463 (+ 6295us) server_negotiation.cc:407] Connection header received
0806 08:57:04.406832 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.406838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.406954 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.407035 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:04.409707 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.410877 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.412945 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.413695 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.413831 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.419123 (+ 5292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:04.419140 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:04.419157 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:04.419201 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:04.422524 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.423256 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.423264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.423266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.423429 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.423857 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.423861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.423863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.424303 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:04.424684 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.424997 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.425231 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 08:57:04.426483 24836 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38064
I20260806 08:57:04.446391 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.429566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.22.98.62:36143)
0806 08:57:04.432189 (+ 2623us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.432193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.432211 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:57:04.432263 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.432267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.432339 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.432433 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:04.434211 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.435302 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.437199 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.437709 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.437827 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.441160 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:04.441165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:04.441169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:04.441207 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:04.443926 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.444663 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.444667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.444669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.444818 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.445185 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.445188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.445190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.445451 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:04.445623 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.445938 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.446101 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2503,"thread_start_us":2376,"threads_started":1}
W20260806 08:57:04.447005 24541 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59360
I20260806 08:57:04.449368 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24649
I20260806 08:57:04.463110 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.450328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.22.98.1:46179)
0806 08:57:04.450511 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.450516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.451311 (+ 795us) server_negotiation.cc:407] Connection header received
0806 08:57:04.451446 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.451454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.451521 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.451649 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:04.452733 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.453560 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.454689 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.455311 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.455429 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.457647 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:04.457652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:04.457655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:04.457685 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:04.460548 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.461165 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.461169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.461171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.461320 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.461668 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.461675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.461677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.461986 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:04.462234 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.462521 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.462768 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 08:57:04.463668 24836 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38076
W20260806 08:57:04.464206 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:04.464376 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 24649
I20260806 08:57:04.493084 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24849
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:04.508669 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.493906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.22.98.2:41721)
0806 08:57:04.494395 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.494400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.494417 (+ 17us) server_negotiation.cc:407] Connection header received
0806 08:57:04.494473 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.494476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.494543 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.494650 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:04.495976 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.496717 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.497870 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.498407 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.498544 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.503202 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:04.503218 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:04.503236 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:04.503281 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:04.506147 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.506809 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.506816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.506819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.506945 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.507233 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.507236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.507238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.507696 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:04.507950 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.508148 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.508329 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":264,"threads_started":1}
W20260806 08:57:04.509357 25044 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41888
W20260806 08:57:04.509997 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24849: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:04.510154 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 24849
I20260806 08:57:04.533186 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25059
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[24478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:04.547431 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:04.533945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.22.98.3:36451)
0806 08:57:04.534496 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:04.534501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:04.534517 (+ 16us) server_negotiation.cc:407] Connection header received
0806 08:57:04.534574 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:04.534578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:04.534638 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:04.534729 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:04.536086 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.536821 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.537975 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:04.538400 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:04.538492 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:04.542455 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:04.542471 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:04.542489 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:04.542522 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:04.545117 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.545702 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.545709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.545711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.545826 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:04.546089 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:04.546092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:04.546094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:04.546478 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:04.546742 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:04.546974 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 08:57:04.547143 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":299,"threads_started":1}
W20260806 08:57:04.548262 25256 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50148
W20260806 08:57:04.548887 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25059: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:04.549013 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 25059
I20260806 08:57:04.575692 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 24498
I20260806 08:57:04.576848 24609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:04.689600 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 24498
2026-08-06T08:57:04Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5113 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[25422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[25422](info): set up 2 sockets
Aug 06 08:57:04 dist-test-slave-jpj1 krb5kdc[25422](info): commencing operation
krb5kdc: starting...
W20260806 08:57:05.914269 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 08:57:06.049175 22920 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 08:57:06.182986 22920 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.134s user 0.001s sys 0.006s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 08:57:06.286121 22920 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.103s user 0.000s sys 0.007s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/test-user.keytab.
W20260806 08:57:06.972088 22920 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.686s user 0.000s sys 0.008s
Aug 06 08:57:06 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:07.076620 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:44569
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:44569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:07.415133 25442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:07.415424 25442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:07.415520 25442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:07.423223 25442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:07.423375 25442 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 08:57:07.423419 25442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:07.423484 25442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:07.423560 25442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:07.423697 25442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:07.434412 25442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:44569
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:44569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:07.435837 25442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:07.437170 25442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:07.449777 25448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:07.449775 25449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:07.450181 25453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:07.450608 25442 server_base.cc:1034] running on GCE node
I20260806 08:57:07.451501 25442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:07.453128 25442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:07.454447 25442 hybrid_clock.cc:648] HybridClock initialized: now 1786006627454282 us; error 80 us; skew 500 ppm
I20260806 08:57:07.454874 25442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:07 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006627, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:07.458189 25442 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:07.459813 25442 webserver.cc:460] Webserver started at http://127.22.98.62:42157/ using document root <none> and password file <none>
I20260806 08:57:07.460565 25442 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:07.460665 25442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:07.461091 25442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:07.464491 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "985ada4bc5714b9aafb7368fd5d561c8"
format_stamp: "Formatted at 2026-08-06 08:57:07 on dist-test-slave-jpj1"
server_key: "d3d4ef4f2f0d27933f62bf1a6941ac8b"
server_key_iv: "fd0ef931f5d909f1e7710edfd827e796"
server_key_version: "encryptionkey@0"
I20260806 08:57:07.465365 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "985ada4bc5714b9aafb7368fd5d561c8"
format_stamp: "Formatted at 2026-08-06 08:57:07 on dist-test-slave-jpj1"
server_key: "d3d4ef4f2f0d27933f62bf1a6941ac8b"
server_key_iv: "fd0ef931f5d909f1e7710edfd827e796"
server_key_version: "encryptionkey@0"
I20260806 08:57:07.471521 25442 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 08:57:07.475524 25463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:07.477140 25442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:07.477329 25442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "985ada4bc5714b9aafb7368fd5d561c8"
format_stamp: "Formatted at 2026-08-06 08:57:07 on dist-test-slave-jpj1"
server_key: "d3d4ef4f2f0d27933f62bf1a6941ac8b"
server_key_iv: "fd0ef931f5d909f1e7710edfd827e796"
server_key_version: "encryptionkey@0"
I20260806 08:57:07.477551 25442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:07.511636 25442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:07.524456 25442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:07.524744 25442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:07.546296 25442 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:44569
I20260806 08:57:07.546303 25554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:44569 every 8 connection(s)
I20260806 08:57:07.547874 25442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:07.549259 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 25442
I20260806 08:57:07.549512 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:07.550361 22920 external_mini_cluster.cc:1428] Setting key f9fec56505270db915489530436b86a1
I20260806 08:57:07.553023 25555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 08:57:07 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:07.568187 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Bootstrap starting.
I20260806 08:57:07.572484 25555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:07.573942 25555 log.cc:824] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:07.576702 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:07.552562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.22.98.62:44569)
0806 08:57:07.553354 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:07.553366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:07.553409 (+ 43us) server_negotiation.cc:407] Connection header received
0806 08:57:07.555040 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:07.555063 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:07.555465 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:07.555865 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:07.557691 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:07.559906 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:07.561134 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:07.562202 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:07.562412 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:07.567460 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:07.567477 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:07.567506 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:07.567591 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:07.571581 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:07.572337 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:07.572351 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:07.572360 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:07.572550 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:07.572987 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:07.572991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:07.572993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:07.573793 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:07.574089 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:07.574561 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 08:57:07.575263 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":370,"threads_started":1}
I20260806 08:57:07.577800 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: No bootstrap required, opened a new log
I20260806 08:57:07.587886 25555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } }
I20260806 08:57:07.588274 25555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:07.588384 25555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 985ada4bc5714b9aafb7368fd5d561c8, State: Initialized, Role: FOLLOWER
I20260806 08:57:07.589051 25555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } }
I20260806 08:57:07.589329 25555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:07.589457 25555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:07.589639 25555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:07.592018 25555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } }
I20260806 08:57:07.592590 25555 leader_election.cc:302] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 985ada4bc5714b9aafb7368fd5d561c8; no voters:
I20260806 08:57:07.593223 25555 leader_election.cc:288] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:07.593355 25561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:07.594056 25561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [term 1 LEADER]: Becoming Leader. State: Replica: 985ada4bc5714b9aafb7368fd5d561c8, State: Running, Role: LEADER
I20260806 08:57:07.594627 25561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } }
I20260806 08:57:07.595513 25555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:07.597678 25562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "985ada4bc5714b9aafb7368fd5d561c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } } }
I20260806 08:57:07.597703 25564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 985ada4bc5714b9aafb7368fd5d561c8. Latest consensus state: current_term: 1 leader_uuid: "985ada4bc5714b9aafb7368fd5d561c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "985ada4bc5714b9aafb7368fd5d561c8" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44569 } } }
I20260806 08:57:07.598157 25562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:07.598157 25564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:07.599658 25567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:07.605036 25567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:07.621918 25567 catalog_manager.cc:1347] Generated new cluster ID: 995228c49fb04c788eeff42a7da80aa3
I20260806 08:57:07.622061 25567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:07.647794 25567 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:07.649194 25567 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:07.654600 25567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Generated new TSK 0
I20260806 08:57:07.655427 25567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:07.715054 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44569
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:08.079010 25591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:08.079304 25591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:08.079408 25591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:08.086895 25591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:08.087025 25591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:08.087117 25591 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 08:57:08.087167 25591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:08.087235 25591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:08.097625 25591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:44569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:08.098950 25591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:08.100414 25591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:08.113513 25603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:08.113474 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:08.113549 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:08.114522 25591 server_base.cc:1034] running on GCE node
I20260806 08:57:08.115156 25591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:08.116138 25591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:08.117372 25591 hybrid_clock.cc:648] HybridClock initialized: now 1786006628117308 us; error 44 us; skew 500 ppm
I20260806 08:57:08.117717 25591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:08 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:08.121269 25591 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:08.122859 25591 webserver.cc:460] Webserver started at http://127.22.98.1:35543/ using document root <none> and password file <none>
I20260806 08:57:08.123646 25591 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:08.123780 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:08.124117 25591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:08.128363 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "5c142761594e4d508e1d62647d7e84bb"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "5c0d9e062e24c9f54a98aad1f3888d5c"
server_key_iv: "dbf405660b303e1a69a28358443b62f7"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.129179 25591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "5c142761594e4d508e1d62647d7e84bb"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "5c0d9e062e24c9f54a98aad1f3888d5c"
server_key_iv: "dbf405660b303e1a69a28358443b62f7"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.134982 25591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 08:57:08.139055 25612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:08.140691 25591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:08.140878 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "5c142761594e4d508e1d62647d7e84bb"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "5c0d9e062e24c9f54a98aad1f3888d5c"
server_key_iv: "dbf405660b303e1a69a28358443b62f7"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.141098 25591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:08.155427 25591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:08.159849 25591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:08.160146 25591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:08.162818 25591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:08.162927 25591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:08.163048 25591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:08.163130 25591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:08 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:08.187906 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.163518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:37981 (local address 127.22.98.62:44569)
0806 08:57:08.164090 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:08.164097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:08.165315 (+ 1218us) server_negotiation.cc:407] Connection header received
0806 08:57:08.167842 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:08.167851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:08.167949 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:08.168038 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:08.170567 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.171578 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.174627 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.175134 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.175280 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.180345 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:08.180353 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:08.180359 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:08.180411 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:08.184238 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.185242 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.185248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.185251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.185435 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.185985 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.185991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.185995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.186295 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:08.186439 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:08.187253 (+ 814us) server_negotiation.cc:299] Negotiation successful
0806 08:57:08.187493 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":310,"threads_started":1}
I20260806 08:57:08.189260 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.164218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.1:37981)
0806 08:57:08.165126 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:08.165176 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:08.167433 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:08.168594 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:08.168614 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:08.169415 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:08.170381 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.170402 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.171810 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.171816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.174417 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.174439 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.175444 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.175455 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.175872 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.176812 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:08.176855 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:08.180028 (+ 3173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:08.184415 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.184425 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.184448 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.185082 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.185582 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.185588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.185591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.185861 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.186460 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:08.186469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:08.186857 (+ 388us) client_negotiation.cc:769] Sending connection context
0806 08:57:08.187133 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 08:57:08.187739 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":377,"threads_started":1}
I20260806 08:57:08.207293 25591 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:33053
I20260806 08:57:08.207314 25799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:33053 every 8 connection(s)
I20260806 08:57:08.209035 25591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:08.211344 25802 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44569
I20260806 08:57:08.211717 25802 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:08.212116 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.196222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:37407 (local address 127.22.98.62:44569)
0806 08:57:08.196732 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:08.196739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:08.197303 (+ 564us) server_negotiation.cc:407] Connection header received
0806 08:57:08.197512 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:08.197516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:08.197582 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:08.197661 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:08.199080 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.200115 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.202382 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.202912 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.203049 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.205383 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:08.205390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:08.205395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:08.205434 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:08.208941 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.209811 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.209818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.209822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.210006 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.210440 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.210446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.210449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.210811 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:08.210962 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:08.211485 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 08:57:08.211777 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370}
I20260806 08:57:08.212327 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.196295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.1:37407)
0806 08:57:08.197160 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:08.197186 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:08.197376 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:08.197806 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:08.197812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:08.198481 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:08.198918 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.198937 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.200263 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.200269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.202213 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.202237 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.203116 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.203127 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.203311 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.203948 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:08.203974 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:08.205168 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:08.209139 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.209144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.209147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.209648 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.210146 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.210152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.210156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.210329 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.210950 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:08.210956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:08.211195 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 08:57:08.211368 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 08:57:08.211814 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":454,"threads_started":1}
I20260806 08:57:08.213071 25802 heartbeater.cc:511] Master 127.22.98.62:44569 requested a full tablet report, sending...
I20260806 08:57:08.215199 25491 ts_manager.cc:194] Registered new tserver with Master: 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053)
I20260806 08:57:08.217185 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:37981
I20260806 08:57:08.219148 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 25591
I20260806 08:57:08.219372 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:08.220086 22920 external_mini_cluster.cc:1428] Setting key 7627b42c040ee3df60b280fbd9a2a776
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:08.264430 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44569
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:08.597931 25807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:08.598248 25807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:08.598382 25807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:08.605804 25807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:08.605916 25807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:08.606034 25807 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 08:57:08.606097 25807 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:08.606200 25807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:08.616220 25807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:44569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:08.617615 25807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:08.618985 25807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:08.631937 25817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:08.632009 25818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:08.632009 25820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:08.632647 25807 server_base.cc:1034] running on GCE node
I20260806 08:57:08.633273 25807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:08.634231 25807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:08.635438 25807 hybrid_clock.cc:648] HybridClock initialized: now 1786006628635391 us; error 36 us; skew 500 ppm
I20260806 08:57:08.635774 25807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:08 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:08.638872 25807 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:08.640319 25807 webserver.cc:460] Webserver started at http://127.22.98.2:36531/ using document root <none> and password file <none>
I20260806 08:57:08.640985 25807 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:08.641106 25807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:08.641410 25807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:08.644377 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "b6af344b05f8d0a8e0afed7f267a2cdf"
server_key_iv: "6f18a803e555c22906d60c788c56d587"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.645140 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "b6af344b05f8d0a8e0afed7f267a2cdf"
server_key_iv: "6f18a803e555c22906d60c788c56d587"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.650224 25807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:08.653642 25829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:08.655107 25807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:08.655267 25807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
format_stamp: "Formatted at 2026-08-06 08:57:08 on dist-test-slave-jpj1"
server_key: "b6af344b05f8d0a8e0afed7f267a2cdf"
server_key_iv: "6f18a803e555c22906d60c788c56d587"
server_key_version: "encryptionkey@0"
I20260806 08:57:08.655447 25807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:08.674930 25807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:08.678810 25807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:08.679083 25807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:08.681805 25807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:08.681919 25807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:08.682040 25807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:08.682117 25807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:08 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:08.706362 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.683074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.2:60207)
0806 08:57:08.683982 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:08.684027 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:08.685934 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:08.686766 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:08.686778 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:08.687677 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:08.688669 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.688701 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.690088 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.690092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.692108 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.692127 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.693241 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.693246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.693427 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.694143 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:08.694165 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:08.697533 (+ 3368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:08.701422 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.701432 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.701448 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.701886 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.702362 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.702367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.702372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.702671 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.703380 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:08.703387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:08.703882 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 08:57:08.704182 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 08:57:08.704805 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":384,"threads_started":1}
I20260806 08:57:08.707149 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.682520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:60207 (local address 127.22.98.62:44569)
0806 08:57:08.682694 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:08.682700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:08.684173 (+ 1473us) server_negotiation.cc:407] Connection header received
0806 08:57:08.686232 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:08.686239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:08.686338 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:08.686445 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:08.688898 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.689870 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.692287 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.693014 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.693162 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.697831 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:08.697839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:08.697845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:08.697893 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:08.701282 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.702025 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.702032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.702036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.702216 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.702855 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.702860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.702863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.703217 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:08.706349 (+ 3132us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:08.706656 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 08:57:08.706901 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 08:57:08.724628 25807 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46261
I20260806 08:57:08.724639 25996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46261 every 8 connection(s)
I20260806 08:57:08.726218 25807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:08.728077 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.713034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:51877 (local address 127.22.98.62:44569)
0806 08:57:08.713226 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:08.713232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:08.714049 (+ 817us) server_negotiation.cc:407] Connection header received
0806 08:57:08.714323 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:08.714329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:08.714411 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:08.714487 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:08.716245 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.717091 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.718993 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.719572 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.719697 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.722195 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:08.722200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:08.722203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:08.722236 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:08.725136 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.725898 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.725901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.725904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.726031 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:08.726423 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:08.726427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:08.726429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:08.726827 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:08.726983 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:08.727503 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 08:57:08.727757 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 08:57:08.728262 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:08.713058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.2:51877)
0806 08:57:08.713918 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:08.713941 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:08.714134 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:08.714720 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:08.714726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:08.715437 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:08.716018 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.716034 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.717285 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.717291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.718852 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:08.718866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:08.719982 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:08.719995 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:08.720140 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:08.720834 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:08.720866 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:08.721938 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:08.725258 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.725263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.725267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.725792 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.726134 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:08.726139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:08.726142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:08.726311 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:08.726971 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:08.726977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:08.727203 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 08:57:08.727369 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 08:57:08.727778 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":459,"threads_started":1}
I20260806 08:57:08.728751 25997 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44569
I20260806 08:57:08.729588 25997 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:08.730505 25997 heartbeater.cc:511] Master 127.22.98.62:44569 requested a full tablet report, sending...
I20260806 08:57:08.732076 25491 ts_manager.cc:194] Registered new tserver with Master: 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261)
I20260806 08:57:08.733104 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:60207
I20260806 08:57:08.735477 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 25807
I20260806 08:57:08.735670 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:08.736099 22920 external_mini_cluster.cc:1428] Setting key 9c851e612fd2fa82ca85c7550c5006f5
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:08.782958 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44569
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:09.115715 26001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:09.116031 26001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:09.116181 26001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:09.123522 26001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:09.123701 26001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:09.123826 26001 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 08:57:09.123893 26001 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:09.123996 26001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:09.134290 26001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:44569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:09.135695 26001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:09.137117 26001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:09.150142 26007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:09.150182 26010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:09.150120 26008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:09.150930 26001 server_base.cc:1034] running on GCE node
I20260806 08:57:09.151579 26001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:09.152597 26001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:09.153803 26001 hybrid_clock.cc:648] HybridClock initialized: now 1786006629153734 us; error 58 us; skew 500 ppm
I20260806 08:57:09.154116 26001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:09 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006629, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:09.157311 26001 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:09.158735 26001 webserver.cc:460] Webserver started at http://127.22.98.3:34019/ using document root <none> and password file <none>
I20260806 08:57:09.159479 26001 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:09.159621 26001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:09.159932 26001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:09.162868 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "a655b21df3b54f899bc9eb9e148a4165"
format_stamp: "Formatted at 2026-08-06 08:57:09 on dist-test-slave-jpj1"
server_key: "1c04519496e8ba17e110a1665ae77530"
server_key_iv: "0c02fca29964cf11ee4457bc6fad24f0"
server_key_version: "encryptionkey@0"
I20260806 08:57:09.163619 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "a655b21df3b54f899bc9eb9e148a4165"
format_stamp: "Formatted at 2026-08-06 08:57:09 on dist-test-slave-jpj1"
server_key: "1c04519496e8ba17e110a1665ae77530"
server_key_iv: "0c02fca29964cf11ee4457bc6fad24f0"
server_key_version: "encryptionkey@0"
I20260806 08:57:09.168817 26001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:57:09.172394 26019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:09.173866 26001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:09.174018 26001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "a655b21df3b54f899bc9eb9e148a4165"
format_stamp: "Formatted at 2026-08-06 08:57:09 on dist-test-slave-jpj1"
server_key: "1c04519496e8ba17e110a1665ae77530"
server_key_iv: "0c02fca29964cf11ee4457bc6fad24f0"
server_key_version: "encryptionkey@0"
I20260806 08:57:09.174193 26001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:09.192766 26001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:09.197166 26001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:09.197453 26001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:09.200423 26001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:09.200529 26001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:09.200646 26001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:09.200748 26001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:09 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006629, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:09.220095 25802 heartbeater.cc:503] Master 127.22.98.62:44569 was elected leader, sending a full tablet report...
I20260806 08:57:09.225152 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.200928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:53315 (local address 127.22.98.62:44569)
0806 08:57:09.201150 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.201157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.202723 (+ 1566us) server_negotiation.cc:407] Connection header received
0806 08:57:09.204995 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.205001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.205097 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.205215 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:09.207783 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.208980 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.211631 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.212365 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.212525 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.217409 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:09.217418 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:09.217424 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:09.217474 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:09.221231 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.222423 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.222428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.222431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.222604 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.223176 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.223182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.223186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.223515 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:09.223785 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.224515 (+ 730us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.224793 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 08:57:09.226334 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.201518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.3:53315)
0806 08:57:09.202515 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.202574 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.204698 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.205572 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.205588 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.206572 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:09.207529 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.207595 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.209241 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.209247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.211433 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.211451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.212663 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.212672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.212968 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.213891 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:09.213934 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:09.217096 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:09.221428 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.221443 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.221463 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.222221 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.222743 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.222749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.222753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.223039 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.223687 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:09.223700 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:09.224127 (+ 427us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.224388 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.224993 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":470,"threads_started":1}
I20260806 08:57:09.248555 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.232973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:35475 (local address 127.22.98.62:44569)
0806 08:57:09.233294 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.233301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.234320 (+ 1019us) server_negotiation.cc:407] Connection header received
0806 08:57:09.234499 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.234506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.234599 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.234711 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:09.236274 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.237279 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.238668 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.239264 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.239404 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.242318 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:09.242326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:09.242331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:09.242388 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:09.245416 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.246324 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.246330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.246332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.246486 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.246853 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.246856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.246858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.247185 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:09.247362 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.247924 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.248177 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260806 08:57:09.248765 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.232961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.22.98.3:35475)
0806 08:57:09.234212 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.234229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.234374 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.234868 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.234874 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.235518 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:09.236026 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.236046 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.237436 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.237442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.238507 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.238520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.239528 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.239574 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.239812 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.240627 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:09.240656 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:09.242075 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:09.245561 (+ 3486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.245567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.245572 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.246198 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.246589 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.246595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.246598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.246762 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.247307 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:09.247312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:09.247577 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.247749 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.248195 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"thread_start_us":856,"threads_started":1}
I20260806 08:57:09.260645 26001 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:39053
I20260806 08:57:09.260694 26193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:39053 every 8 connection(s)
I20260806 08:57:09.261981 26001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:09.264173 26194 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44569
I20260806 08:57:09.264406 26194 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:09.264587 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 26001
I20260806 08:57:09.264746 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:09.265105 26194 heartbeater.cc:511] Master 127.22.98.62:44569 requested a full tablet report, sending...
I20260806 08:57:09.265204 22920 external_mini_cluster.cc:1428] Setting key 362e7bbebcc2903dcb3a8b4c70cd5f1a
I20260806 08:57:09.266774 25490 ts_manager.cc:194] Registered new tserver with Master: a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:09.268131 25490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:53315
I20260806 08:57:09.268785 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:57:09 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 08:57:09 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:09.321858 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.307345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.22.98.62:44569)
0806 08:57:09.307651 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.307658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.307724 (+ 66us) server_negotiation.cc:407] Connection header received
0806 08:57:09.307927 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.307931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.307997 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.308084 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:09.309634 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.310447 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.311730 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.312189 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.312329 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.316212 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:09.316217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:09.316221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:09.316258 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:09.318848 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.319712 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.319717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.319719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.319871 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.320277 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.320282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.320284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.320643 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:09.320898 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.321254 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.321508 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 08:57:09.326153 25490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:09.333189 25490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:09.355811 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.346234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55816 (local address 127.22.98.2:46261)
0806 08:57:09.346747 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.346756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.346786 (+ 30us) server_negotiation.cc:407] Connection header received
0806 08:57:09.346927 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.346935 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.347178 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.347409 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:09.348236 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.349778 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.352605 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.354081 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.354188 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.354421 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.355196 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.355409 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":1098,"threads_started":1}
I20260806 08:57:09.356933 26212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.345625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46261 (local address 127.0.0.1:55816)
0806 08:57:09.346502 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.346554 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.346718 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.347407 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.347414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.347453 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:09.347994 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.348011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.349983 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.349989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.352417 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.352431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.354633 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.354640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.354896 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.354959 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.356360 (+ 1401us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.356557 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":775,"threads_started":1}
I20260806 08:57:09.362332 25899 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb936ebdf08479cb1f771f055403d36 (DEFAULT_TABLE table=test-table [id=933bc32ac8d94f979b44f86f7751126b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:09.363713 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb936ebdf08479cb1f771f055403d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.364820 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.347218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.0.0.1:35146)
0806 08:57:09.348074 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.348090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.348190 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.358661 (+ 10471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.358666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.358691 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:09.359069 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.359083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.360508 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.360513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.362229 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.362250 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.364029 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.364038 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.364208 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.364244 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.364355 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.364489 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":562,"threads_started":1}
I20260806 08:57:09.364904 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.347450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.22.98.1:33053)
0806 08:57:09.358044 (+ 10594us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.358053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.358086 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:57:09.358192 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.358201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.358477 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.358664 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:09.359228 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.360379 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.362388 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.363827 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.363950 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.364161 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.364409 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.364587 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10438,"thread_start_us":10303,"threads_started":1}
I20260806 08:57:09.369073 25685 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb936ebdf08479cb1f771f055403d36 (DEFAULT_TABLE table=test-table [id=933bc32ac8d94f979b44f86f7751126b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:09.370591 25685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb936ebdf08479cb1f771f055403d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.371402 26212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.348131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39053 (local address 127.0.0.1:54848)
0806 08:57:09.357390 (+ 9259us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.357410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.357533 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.358848 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.358853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.358881 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:09.359128 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.359141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.362654 (+ 3513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.362659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.364404 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.364420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.366122 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.366128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.370863 (+ 4735us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.370893 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.370983 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.371135 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9152,"thread_start_us":10253,"threads_started":1}
I20260806 08:57:09.371706 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.348712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.22.98.3:39053)
0806 08:57:09.358190 (+ 9478us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.358197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.358224 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:57:09.358303 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.358308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.358600 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.359317 (+ 717us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:09.360915 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.362508 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.364594 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.365951 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.366085 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.366312 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.371053 (+ 4741us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.371216 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9327,"thread_start_us":9150,"threads_started":1}
I20260806 08:57:09.374830 26086 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb936ebdf08479cb1f771f055403d36 (DEFAULT_TABLE table=test-table [id=933bc32ac8d94f979b44f86f7751126b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:09.376107 26086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb936ebdf08479cb1f771f055403d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.378371 26222 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Bootstrap starting.
I20260806 08:57:09.382233 26222 tablet_bootstrap.cc:654] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.383558 26222 log.cc:824] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:09.384756 26223 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Bootstrap starting.
I20260806 08:57:09.387998 26223 tablet_bootstrap.cc:654] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.388453 26222 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: No bootstrap required, opened a new log
I20260806 08:57:09.388756 26222 ts_tablet_manager.cc:1397] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 08:57:09.389076 26223 log.cc:824] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:09.390421 26227 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Bootstrap starting.
I20260806 08:57:09.392230 26223 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: No bootstrap required, opened a new log
I20260806 08:57:09.392521 26223 ts_tablet_manager.cc:1397] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 08:57:09.394054 26227 tablet_bootstrap.cc:654] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.395254 26227 log.cc:824] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:09.397627 26222 raft_consensus.cc:348] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.397974 26222 raft_consensus.cc:374] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.398075 26222 raft_consensus.cc:729] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.398342 26227 tablet_bootstrap.cc:492] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: No bootstrap required, opened a new log
I20260806 08:57:09.398604 26227 ts_tablet_manager.cc:1397] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 08:57:09.398667 26222 consensus_queue.cc:260] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.399745 25997 heartbeater.cc:503] Master 127.22.98.62:44569 was elected leader, sending a full tablet report...
I20260806 08:57:09.400287 26222 ts_tablet_manager.cc:1428] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260806 08:57:09.402241 26223 raft_consensus.cc:348] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.402671 26223 raft_consensus.cc:374] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.402767 26223 raft_consensus.cc:729] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.403410 26223 consensus_queue.cc:260] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.405452 26223 ts_tablet_manager.cc:1428] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 08:57:09.410594 26227 raft_consensus.cc:348] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.411041 26227 raft_consensus.cc:374] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.411145 26227 raft_consensus.cc:729] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.411813 26227 consensus_queue.cc:260] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.412837 26194 heartbeater.cc:503] Master 127.22.98.62:44569 was elected leader, sending a full tablet report...
I20260806 08:57:09.413398 26227 ts_tablet_manager.cc:1428] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
W20260806 08:57:09.467427 25803 tablet.cc:2369] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:57:09.483700 25998 tablet.cc:2369] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:57:09.513589 26195 tablet.cc:2369] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:09.567245 26231 raft_consensus.cc:482] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:09.567581 26231 raft_consensus.cc:504] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.569352 26231 leader_election.cc:288] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:09.577780 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.569810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:41013 (local address 127.22.98.2:46261)
0806 08:57:09.570000 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.570005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.570611 (+ 606us) server_negotiation.cc:407] Connection header received
0806 08:57:09.570853 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.570859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.570941 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.571040 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:09.571792 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.572820 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.575403 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.576665 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.576803 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.576918 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.577370 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.577517 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 08:57:09.577845 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.570940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:56495 (local address 127.22.98.3:39053)
0806 08:57:09.571144 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.571150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.571362 (+ 212us) server_negotiation.cc:407] Connection header received
0806 08:57:09.571620 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.571625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.571694 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.571801 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:09.572249 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.573169 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.575391 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.576662 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.576802 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.576918 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.577447 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.577572 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 08:57:09.577946 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.570645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39053 (local address 127.22.98.1:56495)
0806 08:57:09.571237 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.571252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.571413 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.571814 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.571817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.571838 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:09.572101 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.572117 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.573294 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.573299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.575247 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.575257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.576938 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.576947 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.577182 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.577234 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.577407 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.577521 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":7,"spinlock_wait_cycles":6784,"thread_start_us":293,"threads_started":1}
I20260806 08:57:09.578262 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.569764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46261 (local address 127.22.98.1:41013)
0806 08:57:09.570478 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.570498 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.570655 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.571174 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.571180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.571216 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:09.571649 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.571661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.572952 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.572956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.575209 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.575228 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.576972 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.576977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.577189 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.577243 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.577342 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.577482 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":67,"spinlock_wait_cycles":4352,"thread_start_us":373,"threads_started":1}
I20260806 08:57:09.578994 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb936ebdf08479cb1f771f055403d36" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165" is_pre_election: true
I20260806 08:57:09.579169 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb936ebdf08479cb1f771f055403d36" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" is_pre_election: true
I20260806 08:57:09.579576 26134 raft_consensus.cc:2454] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:09.579680 25929 raft_consensus.cc:2454] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:09.580240 25616 leader_election.cc:302] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c142761594e4d508e1d62647d7e84bb, a655b21df3b54f899bc9eb9e148a4165; no voters:
I20260806 08:57:09.580610 26231 raft_consensus.cc:2781] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:09.580765 26231 raft_consensus.cc:482] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:09.580876 26231 raft_consensus.cc:3037] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.583272 26231 raft_consensus.cc:504] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.583889 26231 leader_election.cc:288] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Requested vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:09.584378 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb936ebdf08479cb1f771f055403d36" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
I20260806 08:57:09.584425 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb936ebdf08479cb1f771f055403d36" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165"
I20260806 08:57:09.584566 25929 raft_consensus.cc:3037] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.584592 26134 raft_consensus.cc:3037] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.586730 25929 raft_consensus.cc:2454] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:09.586730 26134 raft_consensus.cc:2454] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:09.587106 25616 leader_election.cc:302] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9db61f0c0c48f6b3bbbaf59c80c798, 5c142761594e4d508e1d62647d7e84bb; no voters:
I20260806 08:57:09.587387 26231 raft_consensus.cc:2781] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:09.587867 26231 raft_consensus.cc:686] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 1 LEADER]: Becoming Leader. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Running, Role: LEADER
I20260806 08:57:09.588373 26231 consensus_queue.cc:237] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.593312 25489 catalog_manager.cc:5495] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c142761594e4d508e1d62647d7e84bb (127.22.98.1). New cstate: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } health_report { overall_health: UNKNOWN } } }
Aug 06 08:57:09 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006627, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:09.675395 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.662137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.22.98.62:44569)
0806 08:57:09.662328 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.662332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.662940 (+ 608us) server_negotiation.cc:407] Connection header received
0806 08:57:09.663105 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.663109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.663162 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.663240 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:09.664600 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.665304 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.666520 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.666970 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.667105 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.670307 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:09.670312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:09.670315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:09.670344 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:09.672767 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.673434 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.673440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.673442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.673572 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:09.673926 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:09.673932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:09.673935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:09.674216 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:09.674350 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.674763 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.675067 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":64}
I20260806 08:57:09.675403 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.662156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44569 (local address 127.0.0.1:51232)
0806 08:57:09.662833 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.662849 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.662994 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.663280 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.663284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.664166 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:09.664441 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.664452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.665439 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.665444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.666383 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.666396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.667155 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.667161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.667317 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.667966 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:09.667983 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:09.670129 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:09.672892 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.672905 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.672914 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.673328 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.673672 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:09.673677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:09.673681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:09.673829 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:09.674350 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:09.674359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:09.674518 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.674660 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.674932 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":362,"threads_started":1}
I20260806 08:57:09.680779 25489 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:51232:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 08:57:09.681931 25489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:09.688915 25685 tablet_service.cc:1467] Processing CreateTablet for tablet 7b23791a81f54d6c87f96e51e41b749e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba81e6fb0fe8436684894e2c7b534e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:09.689031 25899 tablet_service.cc:1467] Processing CreateTablet for tablet 7b23791a81f54d6c87f96e51e41b749e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba81e6fb0fe8436684894e2c7b534e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:09.689250 26086 tablet_service.cc:1467] Processing CreateTablet for tablet 7b23791a81f54d6c87f96e51e41b749e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba81e6fb0fe8436684894e2c7b534e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:57:09.689919 25685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b23791a81f54d6c87f96e51e41b749e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.689989 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b23791a81f54d6c87f96e51e41b749e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.689983 26086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b23791a81f54d6c87f96e51e41b749e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:09.695902 26223 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: Bootstrap starting.
I20260806 08:57:09.697260 26222 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: Bootstrap starting.
I20260806 08:57:09.697268 26227 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: Bootstrap starting.
I20260806 08:57:09.697347 26223 tablet_bootstrap.cc:654] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.698922 26227 tablet_bootstrap.cc:654] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.698922 26222 tablet_bootstrap.cc:654] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:09.700656 26223 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: No bootstrap required, opened a new log
I20260806 08:57:09.700819 26223 ts_tablet_manager.cc:1397] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:09.701201 26222 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: No bootstrap required, opened a new log
I20260806 08:57:09.701339 26222 ts_tablet_manager.cc:1397] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 08:57:09.701640 26223 raft_consensus.cc:348] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.701893 26223 raft_consensus.cc:374] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.702001 26223 raft_consensus.cc:729] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.702132 26222 raft_consensus.cc:348] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.702356 26222 raft_consensus.cc:374] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.702291 26223 consensus_queue.cc:260] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.702452 26222 raft_consensus.cc:729] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.702780 26227 tablet_bootstrap.cc:492] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: No bootstrap required, opened a new log
I20260806 08:57:09.702836 26231 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.702989 26227 ts_tablet_manager.cc:1397] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 08:57:09.702903 26222 consensus_queue.cc:260] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.703091 26231 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.703372 26230 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.703831 26230 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.703775 26227 raft_consensus.cc:348] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.704069 26227 raft_consensus.cc:374] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:09.704182 26227 raft_consensus.cc:729] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Initialized, Role: FOLLOWER
I20260806 08:57:09.704455 26227 consensus_queue.cc:260] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.704620 26222 ts_tablet_manager.cc:1428] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:09.704912 26232 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.705318 26232 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.705682 26227 ts_tablet_manager.cc:1428] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:09.706223 26223 ts_tablet_manager.cc:1428] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 08:57:09.840716 26231 raft_consensus.cc:482] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:09.840955 26231 raft_consensus.cc:504] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.841732 26231 leader_election.cc:288] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:09.842231 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b23791a81f54d6c87f96e51e41b749e" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" is_pre_election: true
I20260806 08:57:09.842410 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b23791a81f54d6c87f96e51e41b749e" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165" is_pre_election: true
I20260806 08:57:09.842553 25929 raft_consensus.cc:2454] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:09.842661 26134 raft_consensus.cc:2454] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:09.842998 25616 leader_election.cc:302] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9db61f0c0c48f6b3bbbaf59c80c798, 5c142761594e4d508e1d62647d7e84bb; no voters:
I20260806 08:57:09.843273 26231 raft_consensus.cc:2781] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:09.843382 26231 raft_consensus.cc:482] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:09.843488 26231 raft_consensus.cc:3037] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.845589 26231 raft_consensus.cc:504] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.846125 26231 leader_election.cc:288] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Requested vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:09.846607 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b23791a81f54d6c87f96e51e41b749e" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165"
I20260806 08:57:09.846614 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b23791a81f54d6c87f96e51e41b749e" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
I20260806 08:57:09.846808 26134 raft_consensus.cc:3037] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.846817 25929 raft_consensus.cc:3037] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:09.848783 26134 raft_consensus.cc:2454] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:09.848891 25929 raft_consensus.cc:2454] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:09.849164 25616 leader_election.cc:302] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c142761594e4d508e1d62647d7e84bb, a655b21df3b54f899bc9eb9e148a4165; no voters:
I20260806 08:57:09.849424 26231 raft_consensus.cc:2781] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:09.849614 26231 raft_consensus.cc:686] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [term 1 LEADER]: Becoming Leader. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Running, Role: LEADER
I20260806 08:57:09.849864 26231 consensus_queue.cc:237] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:09.851361 26236 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: TxnStatusTablet state changed. Reason: New leader 5c142761594e4d508e1d62647d7e84bb. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.851732 26236 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:09.852394 26255 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 08:57:09.852697 26255 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 08:57:09.853070 25491 catalog_manager.cc:5495] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c142761594e4d508e1d62647d7e84bb (127.22.98.1). New cstate: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260806 08:57:09.894037 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.887518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.22.98.1:33053)
0806 08:57:09.888124 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.888129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.888146 (+ 17us) server_negotiation.cc:407] Connection header received
0806 08:57:09.888228 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.888232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.888312 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.888442 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:09.888909 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.889743 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.891645 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.892191 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.892288 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.892840 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:09.893448 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:09.893555 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.893700 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.893769 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":302,"threads_started":1}
I20260806 08:57:09.894145 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.887287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.0.0.1:35152)
0806 08:57:09.887577 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.887602 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.887752 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.888499 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.888504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.888517 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:09.888721 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.888734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.889881 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.889887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.891471 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.891489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.892359 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.892366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.892568 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.892646 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:09.893553 (+ 907us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:09.893577 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.893665 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.893836 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 08:57:09.902864 26134 raft_consensus.cc:1261] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:09.903091 25929 raft_consensus.cc:1261] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:09.903934 26231 consensus_queue.cc:1035] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:09.904404 26236 consensus_queue.cc:1035] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:09.912009 26232 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: TxnStatusTablet state changed. Reason: New leader 5c142761594e4d508e1d62647d7e84bb. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.912281 26232 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.915326 26260 mvcc.cc:205] Tried to move back new op lower bound from 7315483156071718912 to 7315483155867209728. Current Snapshot: MvccSnapshot[applied={T|T < 7315483156071718912}]
I20260806 08:57:09.917834 26236 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.918097 26236 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:09.922448 26230 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: TxnStatusTablet state changed. Reason: New leader 5c142761594e4d508e1d62647d7e84bb. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.922730 26230 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.923048 26232 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.923306 26232 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P a655b21df3b54f899bc9eb9e148a4165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.925570 26231 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.925904 26231 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 5c142761594e4d508e1d62647d7e84bb: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:57:09.927347 26263 mvcc.cc:205] Tried to move back new op lower bound from 7315483156071718912 to 7315483155867209728. Current Snapshot: MvccSnapshot[applied={T|T < 7315483156071718912}]
I20260806 08:57:09.929039 26230 tablet_replica.cc:439] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } } }
I20260806 08:57:09.929287 26230 tablet_replica.cc:442] T 7b23791a81f54d6c87f96e51e41b749e P 1b9db61f0c0c48f6b3bbbaf59c80c798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:57:09.938282 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.932890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.22.98.1:33053)
0806 08:57:09.933039 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.933046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.933598 (+ 552us) server_negotiation.cc:407] Connection header received
0806 08:57:09.933797 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.933801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.933863 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.933936 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:09.934385 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.935135 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.936601 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.937071 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.937182 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.937591 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:09.937705 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:09.937788 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.937928 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.938008 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 08:57:09.955739 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.949272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:40517 (local address 127.22.98.1:33053)
0806 08:57:09.949474 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:09.949481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:09.950084 (+ 603us) server_negotiation.cc:407] Connection header received
0806 08:57:09.950303 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:09.950307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:09.950369 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:09.950449 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:09.951104 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.951914 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.953970 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.954406 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.954499 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.955040 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:09.955131 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:09.955197 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:09.955347 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 08:57:09.955436 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":53}
I20260806 08:57:09.956048 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:09.949221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.22.98.1:40517)
0806 08:57:09.949981 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:09.950001 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:09.950163 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:09.950480 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:09.950486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:09.950500 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:09.950938 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.950951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.952054 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.952060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.953834 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:09.953850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:09.954586 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:09.954592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:09.954776 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:09.954850 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:09.955226 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:09.955242 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 08:57:09.955326 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 08:57:09.955497 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":1,"thread_start_us":415,"threads_started":1}
I20260806 08:57:09.965518 26134 raft_consensus.cc:1261] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:09.965673 25929 raft_consensus.cc:1261] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:09.966219 26231 consensus_queue.cc:1035] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:09.966706 26236 consensus_queue.cc:1035] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:10.031075 25489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:10.031386 25489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:10.035218 25489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Sending DeleteTablet for 3 replicas of tablet 7eb936ebdf08479cb1f771f055403d36
I20260806 08:57:10.036412 25899 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb936ebdf08479cb1f771f055403d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:10 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55816
I20260806 08:57:10.036604 25683 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb936ebdf08479cb1f771f055403d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:10 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35146
I20260806 08:57:10.036767 26086 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb936ebdf08479cb1f771f055403d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:10 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:54848
I20260806 08:57:10.038465 25489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:10.039181 25489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:10.048874 26290 tablet_replica.cc:330] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: stopping tablet replica
I20260806 08:57:10.048811 25685 tablet_service.cc:1467] Processing CreateTablet for tablet bb608969c33540619459bf8241dee975 (DEFAULT_TABLE table=test-table [id=c8d2c47b365044a9b05a2c587c23df2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:10.048799 25898 tablet_service.cc:1467] Processing CreateTablet for tablet bb608969c33540619459bf8241dee975 (DEFAULT_TABLE table=test-table [id=c8d2c47b365044a9b05a2c587c23df2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:10.049270 25685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb608969c33540619459bf8241dee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:10.049363 25898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb608969c33540619459bf8241dee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:10.049372 26290 raft_consensus.cc:2229] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:10.050130 26290 raft_consensus.cc:2258] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:10.052040 26085 tablet_service.cc:1467] Processing CreateTablet for tablet bb608969c33540619459bf8241dee975 (DEFAULT_TABLE table=test-table [id=c8d2c47b365044a9b05a2c587c23df2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:10.052556 26085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb608969c33540619459bf8241dee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:10.054421 26290 ts_tablet_manager.cc:1905] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:10.057888 26222 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Bootstrap starting.
I20260806 08:57:10.058842 26289 tablet_replica.cc:330] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: stopping tablet replica
I20260806 08:57:10.059386 26289 raft_consensus.cc:2229] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:10.059769 26222 tablet_bootstrap.cc:654] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:10.059866 26289 raft_consensus.cc:2258] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:10.061992 26289 ts_tablet_manager.cc:1905] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:10.065512 26227 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Bootstrap starting.
I20260806 08:57:10.067207 26227 tablet_bootstrap.cc:654] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:10.067368 26290 ts_tablet_manager.cc:1918] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:10.067564 26290 log.cc:1196] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/7eb936ebdf08479cb1f771f055403d36
I20260806 08:57:10.068109 26290 ts_tablet_manager.cc:1939] T 7eb936ebdf08479cb1f771f055403d36 P 5c142761594e4d508e1d62647d7e84bb: Deleting consensus metadata
I20260806 08:57:10.069597 26289 ts_tablet_manager.cc:1918] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:10.069789 26289 log.cc:1196] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/7eb936ebdf08479cb1f771f055403d36
I20260806 08:57:10.070354 26289 ts_tablet_manager.cc:1939] T 7eb936ebdf08479cb1f771f055403d36 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting consensus metadata
I20260806 08:57:10.071292 25466 catalog_manager.cc:4841] TS 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053): tablet 7eb936ebdf08479cb1f771f055403d36 (table test-table [id=933bc32ac8d94f979b44f86f7751126b]) successfully deleted
I20260806 08:57:10.072029 26223 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Bootstrap starting.
I20260806 08:57:10.073163 25466 catalog_manager.cc:4841] TS 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261): tablet 7eb936ebdf08479cb1f771f055403d36 (table test-table [id=933bc32ac8d94f979b44f86f7751126b]) successfully deleted
I20260806 08:57:10.073657 26222 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: No bootstrap required, opened a new log
I20260806 08:57:10.073688 26223 tablet_bootstrap.cc:654] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:10.073830 26222 ts_tablet_manager.cc:1397] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260806 08:57:10.074617 26222 raft_consensus.cc:348] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.074885 26222 raft_consensus.cc:374] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:10.074970 26222 raft_consensus.cc:729] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Initialized, Role: FOLLOWER
I20260806 08:57:10.075228 26222 consensus_queue.cc:260] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.075977 26222 ts_tablet_manager.cc:1428] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 08:57:10.078853 26223 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: No bootstrap required, opened a new log
I20260806 08:57:10.079075 26223 ts_tablet_manager.cc:1397] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 08:57:10.079717 26227 tablet_bootstrap.cc:492] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: No bootstrap required, opened a new log
I20260806 08:57:10.079981 26227 ts_tablet_manager.cc:1397] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260806 08:57:10.080039 26223 raft_consensus.cc:348] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.080402 26223 raft_consensus.cc:374] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:10.080526 26223 raft_consensus.cc:729] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Initialized, Role: FOLLOWER
I20260806 08:57:10.080845 26223 consensus_queue.cc:260] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.080962 26227 raft_consensus.cc:348] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.081316 26227 raft_consensus.cc:374] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:10.081411 26227 raft_consensus.cc:729] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Initialized, Role: FOLLOWER
I20260806 08:57:10.081664 26227 consensus_queue.cc:260] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.082273 26227 ts_tablet_manager.cc:1428] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:57:10.082387 26291 tablet_replica.cc:330] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: stopping tablet replica
I20260806 08:57:10.082850 26291 raft_consensus.cc:2229] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:10.083185 26291 raft_consensus.cc:2258] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:10.084869 26223 ts_tablet_manager.cc:1428] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 08:57:10.085420 26291 ts_tablet_manager.cc:1905] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:10.092965 26291 ts_tablet_manager.cc:1918] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:10.093142 26291 log.cc:1196] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/7eb936ebdf08479cb1f771f055403d36
I20260806 08:57:10.093680 26291 ts_tablet_manager.cc:1939] T 7eb936ebdf08479cb1f771f055403d36 P a655b21df3b54f899bc9eb9e148a4165: Deleting consensus metadata
I20260806 08:57:10.095516 25466 catalog_manager.cc:4841] TS a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053): tablet 7eb936ebdf08479cb1f771f055403d36 (table test-table [id=933bc32ac8d94f979b44f86f7751126b]) successfully deleted
I20260806 08:57:10.266455 26232 raft_consensus.cc:482] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:10.266770 26232 raft_consensus.cc:504] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.268725 26232 leader_election.cc:288] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053), 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261)
I20260806 08:57:10.277532 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.269208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:43951 (local address 127.22.98.1:33053)
0806 08:57:10.269405 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:10.269412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:10.269954 (+ 542us) server_negotiation.cc:407] Connection header received
0806 08:57:10.270263 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:10.270269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:10.270343 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:10.270398 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:10.271247 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.272323 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.274931 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.276305 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.276401 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.276513 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:10.277101 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 08:57:10.277250 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 08:57:10.278589 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.268945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.22.98.3:43951)
0806 08:57:10.269841 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:10.269860 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:10.270097 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:10.270589 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:10.270593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:10.270632 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:10.271040 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.271053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.272524 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.272529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.274736 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.274753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.276700 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.276706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.276915 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.276950 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 08:57:10.278063 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0806 08:57:10.278216 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":616,"threads_started":1}
I20260806 08:57:10.280347 25719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb608969c33540619459bf8241dee975" candidate_uuid: "a655b21df3b54f899bc9eb9e148a4165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c142761594e4d508e1d62647d7e84bb" is_pre_election: true
I20260806 08:57:10.280901 25719 raft_consensus.cc:2454] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a655b21df3b54f899bc9eb9e148a4165 in term 0.
I20260806 08:57:10.281638 26022 leader_election.cc:302] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c142761594e4d508e1d62647d7e84bb, a655b21df3b54f899bc9eb9e148a4165; no voters:
I20260806 08:57:10.282066 26232 raft_consensus.cc:2781] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:10.282191 26232 raft_consensus.cc:482] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:10.282290 26232 raft_consensus.cc:3037] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:10.285710 26232 raft_consensus.cc:504] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.286384 26232 leader_election.cc:288] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [CANDIDATE]: Term 1 election: Requested vote from peers 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053), 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261)
I20260806 08:57:10.287142 25719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb608969c33540619459bf8241dee975" candidate_uuid: "a655b21df3b54f899bc9eb9e148a4165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c142761594e4d508e1d62647d7e84bb"
I20260806 08:57:10.287340 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.269950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46261 (local address 127.22.98.3:51615)
0806 08:57:10.278998 (+ 9048us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:10.279017 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:10.279145 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:10.280267 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:10.280271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:10.280299 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:10.280468 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.280479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.282712 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.282718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.284751 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.284769 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.286478 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.286484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.286779 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.286835 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 08:57:10.286911 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 08:57:10.287065 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8945,"thread_start_us":9405,"threads_started":1}
I20260806 08:57:10.287349 25719 raft_consensus.cc:3037] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:10.288403 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.270324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:51615 (local address 127.22.98.2:46261)
0806 08:57:10.279925 (+ 9601us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:10.279932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:10.279958 (+ 26us) server_negotiation.cc:407] Connection header received
0806 08:57:10.280029 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:10.280034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:10.280109 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:10.280206 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:10.280617 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.282477 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.284982 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.286264 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.287800 (+ 1536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.287898 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:10.287949 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 08:57:10.288083 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9473,"thread_start_us":9682,"threads_started":1}
I20260806 08:57:10.290710 25719 raft_consensus.cc:2454] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a655b21df3b54f899bc9eb9e148a4165 in term 1.
I20260806 08:57:10.291281 26022 leader_election.cc:302] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c142761594e4d508e1d62647d7e84bb, a655b21df3b54f899bc9eb9e148a4165; no voters:
I20260806 08:57:10.291626 26232 raft_consensus.cc:2781] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:10.292117 25928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb608969c33540619459bf8241dee975" candidate_uuid: "a655b21df3b54f899bc9eb9e148a4165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" is_pre_election: true
I20260806 08:57:10.292405 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb608969c33540619459bf8241dee975" candidate_uuid: "a655b21df3b54f899bc9eb9e148a4165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
I20260806 08:57:10.292464 25928 raft_consensus.cc:2454] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a655b21df3b54f899bc9eb9e148a4165 in term 0.
I20260806 08:57:10.293406 26232 raft_consensus.cc:686] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 1 LEADER]: Becoming Leader. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Running, Role: LEADER
I20260806 08:57:10.293893 26232 consensus_queue.cc:237] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:10.297359 25489 catalog_manager.cc:5495] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 reported cstate change: term changed from 0 to 1, leader changed from <none> to a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "a655b21df3b54f899bc9eb9e148a4165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } health_report { overall_health: HEALTHY } } }
I20260806 08:57:10.321698 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.315936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.22.98.3:39053)
0806 08:57:10.316495 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:10.316499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:10.316517 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:57:10.316602 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:10.316605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:10.316676 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:10.316836 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:10.317269 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.318034 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.319348 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.319794 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.319959 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.320414 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:10.321036 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:10.321173 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:10.321282 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 08:57:10.321366 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":303,"threads_started":1}
I20260806 08:57:10.339165 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.332648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.22.98.3:54137)
0806 08:57:10.333385 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:10.333401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:10.333525 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:10.333970 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:10.333974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:10.333990 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:10.334318 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.334327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.335832 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.335837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.336927 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.336941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.337960 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.337965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.338098 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.338165 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:10.338660 (+ 495us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:10.338670 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 08:57:10.338753 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 08:57:10.338864 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":377,"threads_started":1}
I20260806 08:57:10.339190 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.332742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:54137 (local address 127.22.98.1:33053)
0806 08:57:10.332947 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:10.332954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:10.333488 (+ 534us) server_negotiation.cc:407] Connection header received
0806 08:57:10.333665 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:10.333671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:10.333779 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:10.333890 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:10.334485 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.335690 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.337077 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.337781 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.337923 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.338361 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:10.338545 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:10.338639 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:10.338773 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 08:57:10.338875 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:57:10.348805 25929 raft_consensus.cc:3037] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:10.349334 25719 raft_consensus.cc:1261] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Refusing update from remote peer a655b21df3b54f899bc9eb9e148a4165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:10.350047 26301 consensus_queue.cc:1035] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:10.352198 25929 raft_consensus.cc:1261] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Refusing update from remote peer a655b21df3b54f899bc9eb9e148a4165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:10.353081 26301 consensus_queue.cc:1035] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:10.356681 26261 mvcc.cc:205] Tried to move back new op lower bound from 7315483157902426112 to 7315483157687660544. Current Snapshot: MvccSnapshot[applied={T|T < 7315483157902426112}]
I20260806 08:57:10.383124 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.376951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:40063 (local address 127.22.98.3:39053)
0806 08:57:10.377612 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:10.377617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:10.377630 (+ 13us) server_negotiation.cc:407] Connection header received
0806 08:57:10.377701 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:10.377705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:10.377788 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:10.377861 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:10.378342 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.379234 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.381262 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.381870 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.381994 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.382434 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:10.382533 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:10.382598 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:10.382830 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 08:57:10.382911 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565}
I20260806 08:57:10.383270 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:10.376859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39053 (local address 127.22.98.1:40063)
0806 08:57:10.377270 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:10.377290 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:10.377422 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:10.377915 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:10.377920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:10.377932 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:10.378156 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.378175 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.379371 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.379377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.381085 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:10.381107 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:10.382019 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:10.382026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:10.382226 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:10.382280 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:10.382628 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:10.382661 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 08:57:10.382746 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 08:57:10.382892 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
W20260806 08:57:10.384483 26261 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 08:57:11.990335 25490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:11.990576 25490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:11.994971 25490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Sending DeleteTablet for 3 replicas of tablet bb608969c33540619459bf8241dee975
I20260806 08:57:11.995870 25685 tablet_service.cc:1514] Processing DeleteTablet for tablet bb608969c33540619459bf8241dee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35146
I20260806 08:57:11.995986 26086 tablet_service.cc:1514] Processing DeleteTablet for tablet bb608969c33540619459bf8241dee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:54848
I20260806 08:57:11.996019 25899 tablet_service.cc:1514] Processing DeleteTablet for tablet bb608969c33540619459bf8241dee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55816
I20260806 08:57:12.008507 26349 tablet_replica.cc:330] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: stopping tablet replica
I20260806 08:57:12.008888 26349 raft_consensus.cc:2229] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:12.009205 26349 raft_consensus.cc:2258] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.010798 26349 ts_tablet_manager.cc:1905] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.011202 26348 tablet_replica.cc:330] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: stopping tablet replica
I20260806 08:57:12.011423 26347 tablet_replica.cc:330] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: stopping tablet replica
I20260806 08:57:12.011441 26348 raft_consensus.cc:2229] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:12.011660 26347 raft_consensus.cc:2229] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:12.011940 26347 raft_consensus.cc:2258] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.012143 26348 raft_consensus.cc:2258] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.013129 26348 ts_tablet_manager.cc:1905] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.013248 26347 ts_tablet_manager.cc:1905] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.018136 26349 ts_tablet_manager.cc:1918] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:12.018347 26349 log.cc:1196] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/bb608969c33540619459bf8241dee975
I20260806 08:57:12.018812 26349 ts_tablet_manager.cc:1939] T bb608969c33540619459bf8241dee975 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting consensus metadata
I20260806 08:57:12.020154 25466 catalog_manager.cc:4841] TS 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261): tablet bb608969c33540619459bf8241dee975 (table test-table [id=c8d2c47b365044a9b05a2c587c23df2b]) successfully deleted
I20260806 08:57:12.020308 26348 ts_tablet_manager.cc:1918] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:12.020301 26347 ts_tablet_manager.cc:1918] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:12.020447 26347 log.cc:1196] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/bb608969c33540619459bf8241dee975
I20260806 08:57:12.020447 26348 log.cc:1196] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/bb608969c33540619459bf8241dee975
I20260806 08:57:12.020912 26348 ts_tablet_manager.cc:1939] T bb608969c33540619459bf8241dee975 P a655b21df3b54f899bc9eb9e148a4165: Deleting consensus metadata
I20260806 08:57:12.020973 26347 ts_tablet_manager.cc:1939] T bb608969c33540619459bf8241dee975 P 5c142761594e4d508e1d62647d7e84bb: Deleting consensus metadata
I20260806 08:57:12.022056 25466 catalog_manager.cc:4841] TS 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053): tablet bb608969c33540619459bf8241dee975 (table test-table [id=c8d2c47b365044a9b05a2c587c23df2b]) successfully deleted
I20260806 08:57:12.022324 25466 catalog_manager.cc:4841] TS a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053): tablet bb608969c33540619459bf8241dee975 (table test-table [id=c8d2c47b365044a9b05a2c587c23df2b]) successfully deleted
Aug 06 08:57:12 dist-test-slave-jpj1 krb5kdc[25422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 08:57:12 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:12.062721 26359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.047416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.22.98.62:44569)
0806 08:57:12.048458 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.048463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.048502 (+ 39us) server_negotiation.cc:407] Connection header received
0806 08:57:12.048588 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.048592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.048666 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.048771 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:12.050362 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.051334 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.052717 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.053130 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.053231 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.057268 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:12.057273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:12.057277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:12.057314 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:12.060354 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:12.061014 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:12.061019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:12.061021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:12.061185 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:12.061485 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:12.061488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:12.061490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:12.061802 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:12.061990 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.062262 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.062443 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":623,"threads_started":1}
I20260806 08:57:12.066856 25490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:12.067503 25490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:12.074905 25685 tablet_service.cc:1467] Processing CreateTablet for tablet a97484b306174044b5d242a9e479eabd (DEFAULT_TABLE table=test-table [id=2c8b0b7b94d94e32837195424f7edea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.074905 26086 tablet_service.cc:1467] Processing CreateTablet for tablet a97484b306174044b5d242a9e479eabd (DEFAULT_TABLE table=test-table [id=2c8b0b7b94d94e32837195424f7edea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.074928 25899 tablet_service.cc:1467] Processing CreateTablet for tablet a97484b306174044b5d242a9e479eabd (DEFAULT_TABLE table=test-table [id=2c8b0b7b94d94e32837195424f7edea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.075408 26086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97484b306174044b5d242a9e479eabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.075412 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97484b306174044b5d242a9e479eabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.075443 25685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97484b306174044b5d242a9e479eabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.082877 26360 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Bootstrap starting.
I20260806 08:57:12.083309 26361 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Bootstrap starting.
I20260806 08:57:12.084786 26360 tablet_bootstrap.cc:654] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.085261 26361 tablet_bootstrap.cc:654] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.086138 26362 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Bootstrap starting.
I20260806 08:57:12.087589 26360 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: No bootstrap required, opened a new log
I20260806 08:57:12.087769 26360 ts_tablet_manager.cc:1397] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 08:57:12.087824 26362 tablet_bootstrap.cc:654] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.087881 26361 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: No bootstrap required, opened a new log
I20260806 08:57:12.088054 26361 ts_tablet_manager.cc:1397] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:12.088641 26360 raft_consensus.cc:348] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.088917 26360 raft_consensus.cc:374] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.089018 26360 raft_consensus.cc:729] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.089087 26361 raft_consensus.cc:348] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.089336 26361 raft_consensus.cc:374] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.089257 26360 consensus_queue.cc:260] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.089468 26361 raft_consensus.cc:729] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.089845 26361 consensus_queue.cc:260] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.089990 26360 ts_tablet_manager.cc:1428] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 08:57:12.091953 26361 ts_tablet_manager.cc:1428] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260806 08:57:12.092880 26362 tablet_bootstrap.cc:492] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: No bootstrap required, opened a new log
I20260806 08:57:12.093134 26362 ts_tablet_manager.cc:1397] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 08:57:12.093974 26362 raft_consensus.cc:348] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.094216 26362 raft_consensus.cc:374] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.094319 26362 raft_consensus.cc:729] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.094578 26362 consensus_queue.cc:260] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.095117 26362 ts_tablet_manager.cc:1428] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 08:57:12.194917 26317 raft_consensus.cc:482] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:12.195147 26317 raft_consensus.cc:504] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.195971 26317 leader_election.cc:288] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:12.196626 25928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97484b306174044b5d242a9e479eabd" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" is_pre_election: true
I20260806 08:57:12.196864 25928 raft_consensus.cc:2454] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:12.196784 26132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97484b306174044b5d242a9e479eabd" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165" is_pre_election: true
I20260806 08:57:12.197062 26132 raft_consensus.cc:2454] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 0.
I20260806 08:57:12.197422 25616 leader_election.cc:302] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9db61f0c0c48f6b3bbbaf59c80c798, 5c142761594e4d508e1d62647d7e84bb; no voters:
I20260806 08:57:12.197788 26317 raft_consensus.cc:2781] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:12.197911 26317 raft_consensus.cc:482] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:12.198005 26317 raft_consensus.cc:3037] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.200253 26317 raft_consensus.cc:504] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.200796 26317 leader_election.cc:288] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Requested vote from peers 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:12.201448 26132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97484b306174044b5d242a9e479eabd" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165"
I20260806 08:57:12.201542 25928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97484b306174044b5d242a9e479eabd" candidate_uuid: "5c142761594e4d508e1d62647d7e84bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798"
I20260806 08:57:12.201691 26132 raft_consensus.cc:3037] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.201838 25928 raft_consensus.cc:3037] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.204308 26132 raft_consensus.cc:2454] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:12.204331 25928 raft_consensus.cc:2454] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c142761594e4d508e1d62647d7e84bb in term 1.
I20260806 08:57:12.204807 25616 leader_election.cc:302] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c142761594e4d508e1d62647d7e84bb, a655b21df3b54f899bc9eb9e148a4165; no voters:
I20260806 08:57:12.205101 26317 raft_consensus.cc:2781] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:12.205246 26317 raft_consensus.cc:686] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 1 LEADER]: Becoming Leader. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Running, Role: LEADER
I20260806 08:57:12.205686 26317 consensus_queue.cc:237] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.208916 25490 catalog_manager.cc:5495] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c142761594e4d508e1d62647d7e84bb (127.22.98.1). New cstate: current_term: 1 leader_uuid: "5c142761594e4d508e1d62647d7e84bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260806 08:57:12.253285 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.247949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.22.98.1:33053)
0806 08:57:12.248600 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.248605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.248645 (+ 40us) server_negotiation.cc:407] Connection header received
0806 08:57:12.248704 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.248708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.248785 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.248871 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:12.249302 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.250127 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.251479 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.252028 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.252126 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.252562 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.252674 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.252768 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.252935 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.253003 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":307,"threads_started":1}
I20260806 08:57:12.263123 25928 raft_consensus.cc:1261] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:12.263145 26132 raft_consensus.cc:1261] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Refusing update from remote peer 5c142761594e4d508e1d62647d7e84bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:12.263722 26236 consensus_queue.cc:1035] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:12.264009 26317 consensus_queue.cc:1035] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:12.316704 25489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:12.317102 25489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:12.320130 25489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Sending DeleteTablet for 3 replicas of tablet a97484b306174044b5d242a9e479eabd
I20260806 08:57:12.320894 25683 tablet_service.cc:1514] Processing DeleteTablet for tablet a97484b306174044b5d242a9e479eabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:12 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35146
I20260806 08:57:12.321097 26086 tablet_service.cc:1514] Processing DeleteTablet for tablet a97484b306174044b5d242a9e479eabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:12 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:54848
I20260806 08:57:12.321097 25899 tablet_service.cc:1514] Processing DeleteTablet for tablet a97484b306174044b5d242a9e479eabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:12 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55816
I20260806 08:57:12.321316 26347 tablet_replica.cc:330] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: stopping tablet replica
I20260806 08:57:12.321467 26348 tablet_replica.cc:330] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: stopping tablet replica
I20260806 08:57:12.321631 26349 tablet_replica.cc:330] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: stopping tablet replica
I20260806 08:57:12.321681 26347 raft_consensus.cc:2229] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:12.321744 26348 raft_consensus.cc:2229] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:12.321938 26348 raft_consensus.cc:2258] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.321944 26349 raft_consensus.cc:2229] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:12.322018 26347 raft_consensus.cc:2258] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.322154 26349 raft_consensus.cc:2258] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:12.322382 25489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:12.323045 25489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:12.323194 26348 ts_tablet_manager.cc:1905] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.323251 26347 ts_tablet_manager.cc:1905] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.325028 26349 ts_tablet_manager.cc:1905] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:12.328969 26347 ts_tablet_manager.cc:1918] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:12.329216 26347 log.cc:1196] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/a97484b306174044b5d242a9e479eabd
I20260806 08:57:12.329818 26347 ts_tablet_manager.cc:1939] T a97484b306174044b5d242a9e479eabd P 5c142761594e4d508e1d62647d7e84bb: Deleting consensus metadata
I20260806 08:57:12.330021 26348 ts_tablet_manager.cc:1918] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:12.330185 26348 log.cc:1196] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/a97484b306174044b5d242a9e479eabd
I20260806 08:57:12.330600 26348 ts_tablet_manager.cc:1939] T a97484b306174044b5d242a9e479eabd P a655b21df3b54f899bc9eb9e148a4165: Deleting consensus metadata
I20260806 08:57:12.331156 25466 catalog_manager.cc:4841] TS 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053): tablet a97484b306174044b5d242a9e479eabd (table test-table [id=2c8b0b7b94d94e32837195424f7edea2]) successfully deleted
I20260806 08:57:12.331504 26349 ts_tablet_manager.cc:1918] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:57:12.331707 26349 log.cc:1196] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/a97484b306174044b5d242a9e479eabd
I20260806 08:57:12.332079 26349 ts_tablet_manager.cc:1939] T a97484b306174044b5d242a9e479eabd P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting consensus metadata
I20260806 08:57:12.332211 25466 catalog_manager.cc:4841] TS a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053): tablet a97484b306174044b5d242a9e479eabd (table test-table [id=2c8b0b7b94d94e32837195424f7edea2]) successfully deleted
I20260806 08:57:12.333122 25683 tablet_service.cc:1467] Processing CreateTablet for tablet f2d1d85a865846879791063b10cb9941 (DEFAULT_TABLE table=test-table [id=4bc48138b8ce45f9b081ab3770088048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.333246 25899 tablet_service.cc:1467] Processing CreateTablet for tablet f2d1d85a865846879791063b10cb9941 (DEFAULT_TABLE table=test-table [id=4bc48138b8ce45f9b081ab3770088048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.333359 26086 tablet_service.cc:1467] Processing CreateTablet for tablet f2d1d85a865846879791063b10cb9941 (DEFAULT_TABLE table=test-table [id=4bc48138b8ce45f9b081ab3770088048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:12.333621 25683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d1d85a865846879791063b10cb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.333665 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d1d85a865846879791063b10cb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.333740 26086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d1d85a865846879791063b10cb9941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:12.334025 25466 catalog_manager.cc:4841] TS 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261): tablet a97484b306174044b5d242a9e479eabd (table test-table [id=2c8b0b7b94d94e32837195424f7edea2]) successfully deleted
I20260806 08:57:12.339419 26360 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Bootstrap starting.
I20260806 08:57:12.340058 26362 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Bootstrap starting.
I20260806 08:57:12.340304 26361 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Bootstrap starting.
I20260806 08:57:12.341178 26360 tablet_bootstrap.cc:654] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.341791 26362 tablet_bootstrap.cc:654] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.342053 26361 tablet_bootstrap.cc:654] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:12.343847 26360 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: No bootstrap required, opened a new log
I20260806 08:57:12.344139 26360 ts_tablet_manager.cc:1397] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:12.345024 26360 raft_consensus.cc:348] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.345367 26360 raft_consensus.cc:374] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.345561 26360 raft_consensus.cc:729] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c142761594e4d508e1d62647d7e84bb, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.345882 26360 consensus_queue.cc:260] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.346603 26360 ts_tablet_manager.cc:1428] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 08:57:12.348655 26361 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: No bootstrap required, opened a new log
I20260806 08:57:12.348824 26362 tablet_bootstrap.cc:492] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: No bootstrap required, opened a new log
I20260806 08:57:12.348843 26361 ts_tablet_manager.cc:1397] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260806 08:57:12.349001 26362 ts_tablet_manager.cc:1397] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 08:57:12.349809 26362 raft_consensus.cc:348] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.349809 26361 raft_consensus.cc:348] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.350068 26362 raft_consensus.cc:374] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.350068 26361 raft_consensus.cc:374] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:12.350143 26362 raft_consensus.cc:729] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655b21df3b54f899bc9eb9e148a4165, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.350178 26361 raft_consensus.cc:729] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Initialized, Role: FOLLOWER
I20260806 08:57:12.350395 26362 consensus_queue.cc:260] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.350395 26361 consensus_queue.cc:260] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.350940 26361 ts_tablet_manager.cc:1428] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 08:57:12.351648 26362 ts_tablet_manager.cc:1428] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:12.657567 26367 raft_consensus.cc:482] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:12.657819 26367 raft_consensus.cc:504] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.659471 26367 leader_election.cc:288] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:12.666353 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.660007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:48417 (local address 127.22.98.1:33053)
0806 08:57:12.660184 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.660191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.660422 (+ 231us) server_negotiation.cc:407] Connection header received
0806 08:57:12.661121 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.661126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.661190 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.661264 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:12.661976 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.662717 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.664687 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.665491 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.665604 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.665710 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.665956 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.666084 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 08:57:12.666538 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.659743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.22.98.2:48417)
0806 08:57:12.660310 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:12.660328 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:12.660949 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:12.661305 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:12.661309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:12.661335 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:12.661787 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.661799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.662889 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.662893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.664535 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.664547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.665665 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.665670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.665809 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.665833 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 08:57:12.665914 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 08:57:12.666053 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":30,"thread_start_us":265,"threads_started":1}
I20260806 08:57:12.666879 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.660574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:57029 (local address 127.22.98.3:39053)
0806 08:57:12.661043 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.661049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.661152 (+ 103us) server_negotiation.cc:407] Connection header received
0806 08:57:12.661342 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.661347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.661406 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.661489 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:12.662434 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.663166 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.664920 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.665780 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.665895 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.666000 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.666349 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.666479 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":217,"threads_started":1}
I20260806 08:57:12.667210 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.660426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39053 (local address 127.22.98.2:57029)
0806 08:57:12.661055 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:12.661073 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:12.661214 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:12.661536 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:12.661540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:12.661571 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:12.662264 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.662276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.663301 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.663306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.664718 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.664731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.666004 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.666011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.666190 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.666221 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 08:57:12.666317 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 08:57:12.666450 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":6,"thread_start_us":259,"threads_started":1}
I20260806 08:57:12.667336 25719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d1d85a865846879791063b10cb9941" candidate_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c142761594e4d508e1d62647d7e84bb" is_pre_election: true
I20260806 08:57:12.667647 25719 raft_consensus.cc:2454] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b9db61f0c0c48f6b3bbbaf59c80c798 in term 0.
I20260806 08:57:12.667989 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d1d85a865846879791063b10cb9941" candidate_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165" is_pre_election: true
I20260806 08:57:12.668202 25833 leader_election.cc:302] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9db61f0c0c48f6b3bbbaf59c80c798, 5c142761594e4d508e1d62647d7e84bb; no voters:
I20260806 08:57:12.668316 26134 raft_consensus.cc:2454] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b9db61f0c0c48f6b3bbbaf59c80c798 in term 0.
I20260806 08:57:12.668663 26367 raft_consensus.cc:2781] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:12.668795 26367 raft_consensus.cc:482] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:12.668905 26367 raft_consensus.cc:3037] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.671139 26367 raft_consensus.cc:504] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.671762 26367 leader_election.cc:288] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [CANDIDATE]: Term 1 election: Requested vote from peers 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053), a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053)
I20260806 08:57:12.672438 25719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d1d85a865846879791063b10cb9941" candidate_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c142761594e4d508e1d62647d7e84bb"
I20260806 08:57:12.672623 25719 raft_consensus.cc:3037] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.673029 26134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d1d85a865846879791063b10cb9941" candidate_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a655b21df3b54f899bc9eb9e148a4165"
I20260806 08:57:12.673290 26134 raft_consensus.cc:3037] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:12.675575 25719 raft_consensus.cc:2454] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b9db61f0c0c48f6b3bbbaf59c80c798 in term 1.
I20260806 08:57:12.676110 25833 leader_election.cc:302] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b9db61f0c0c48f6b3bbbaf59c80c798, 5c142761594e4d508e1d62647d7e84bb; no voters:
I20260806 08:57:12.676157 26134 raft_consensus.cc:2454] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b9db61f0c0c48f6b3bbbaf59c80c798 in term 1.
I20260806 08:57:12.676478 26367 raft_consensus.cc:2781] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:12.677006 26367 raft_consensus.cc:686] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 LEADER]: Becoming Leader. State: Replica: 1b9db61f0c0c48f6b3bbbaf59c80c798, State: Running, Role: LEADER
I20260806 08:57:12.677398 26367 consensus_queue.cc:237] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } }
I20260806 08:57:12.681161 25490 catalog_manager.cc:5495] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b9db61f0c0c48f6b3bbbaf59c80c798" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260806 08:57:12.724062 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.717712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.22.98.2:46261)
0806 08:57:12.718429 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.718433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.718452 (+ 19us) server_negotiation.cc:407] Connection header received
0806 08:57:12.718534 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.718538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.718611 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.718746 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:12.719346 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.720185 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.721638 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.722113 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.722234 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.722691 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.723387 (+ 696us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.723495 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.723651 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.723717 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":472,"threads_started":1}
I20260806 08:57:12.742182 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.735335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:57695 (local address 127.22.98.1:33053)
0806 08:57:12.735566 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.735572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.736418 (+ 846us) server_negotiation.cc:407] Connection header received
0806 08:57:12.736582 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.736589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.736686 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.736781 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:12.737460 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.738640 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.740167 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.740875 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.741000 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.741463 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.741593 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.741677 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.741774 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.741870 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 08:57:12.742241 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.735226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33053 (local address 127.22.98.2:57695)
0806 08:57:12.736321 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:12.736338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:12.736455 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:12.736823 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:12.736826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:12.736836 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:12.737314 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.737325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.738824 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.738828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.740029 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.740042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.741039 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.741043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.741202 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.741261 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.741673 (+ 412us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.741687 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 08:57:12.741740 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 08:57:12.741866 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":711,"threads_started":1}
I20260806 08:57:12.751312 25719 raft_consensus.cc:1261] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Refusing update from remote peer 1b9db61f0c0c48f6b3bbbaf59c80c798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:12.751314 26134 raft_consensus.cc:1261] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Refusing update from remote peer 1b9db61f0c0c48f6b3bbbaf59c80c798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:12.752172 26381 consensus_queue.cc:1035] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c142761594e4d508e1d62647d7e84bb" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 08:57:12.753224 26381 consensus_queue.cc:1035] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a655b21df3b54f899bc9eb9e148a4165" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:12.779639 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.773741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54887 (local address 127.22.98.2:46261)
0806 08:57:12.773926 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:12.773933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:12.774369 (+ 436us) server_negotiation.cc:407] Connection header received
0806 08:57:12.774550 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:12.774555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:12.774636 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:12.774738 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:12.775271 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.776127 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.778041 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.778533 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.778630 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.779005 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.779104 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.779171 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:12.779259 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 08:57:12.779325 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 08:57:12.779898 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:12.773606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46261 (local address 127.22.98.1:54887)
0806 08:57:12.774250 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:12.774270 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:12.774421 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:12.774746 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:12.774749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:12.774759 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:57:12.775129 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.775140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.776255 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.776260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.777895 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:12.777911 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:12.778682 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:12.778687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:12.778832 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:12.778872 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:57:12.779174 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:57:12.779182 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 08:57:12.779234 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 08:57:12.779343 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":324,"threads_started":1}
W20260806 08:57:12.781157 26319 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 08:57:14.383472 25489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:14.383807 25489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:14.388489 25489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 985ada4bc5714b9aafb7368fd5d561c8: Sending DeleteTablet for 3 replicas of tablet f2d1d85a865846879791063b10cb9941
I20260806 08:57:14.389482 25683 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d1d85a865846879791063b10cb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:14 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35146
I20260806 08:57:14.389523 25899 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d1d85a865846879791063b10cb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:14 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55816
I20260806 08:57:14.389631 26086 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d1d85a865846879791063b10cb9941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:14 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:54848
I20260806 08:57:14.390252 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25591
I20260806 08:57:14.393710 26431 tablet_replica.cc:330] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: stopping tablet replica
I20260806 08:57:14.394009 26431 raft_consensus.cc:2229] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:14.394366 26431 raft_consensus.cc:2258] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:14.395037 26433 tablet_replica.cc:330] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: stopping tablet replica
I20260806 08:57:14.395849 26432 tablet_replica.cc:330] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: stopping tablet replica
I20260806 08:57:14.395964 26431 ts_tablet_manager.cc:1905] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:14.396075 26432 raft_consensus.cc:2229] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:14.396302 26433 raft_consensus.cc:2229] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:14.396375 26432 raft_consensus.cc:2258] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:14.397039 26433 raft_consensus.cc:2258] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:14.397770 26432 ts_tablet_manager.cc:1905] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:14.398128 26433 ts_tablet_manager.cc:1905] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:14.403419 26431 ts_tablet_manager.cc:1918] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:14.403596 26431 log.cc:1196] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/f2d1d85a865846879791063b10cb9941
I20260806 08:57:14.404129 26431 ts_tablet_manager.cc:1939] T f2d1d85a865846879791063b10cb9941 P 5c142761594e4d508e1d62647d7e84bb: Deleting consensus metadata
I20260806 08:57:14.404335 26433 ts_tablet_manager.cc:1918] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:14.404466 26433 log.cc:1196] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/f2d1d85a865846879791063b10cb9941
I20260806 08:57:14.404773 26432 ts_tablet_manager.cc:1918] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 08:57:14.404842 26433 ts_tablet_manager.cc:1939] T f2d1d85a865846879791063b10cb9941 P 1b9db61f0c0c48f6b3bbbaf59c80c798: Deleting consensus metadata
I20260806 08:57:14.404917 26432 log.cc:1196] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/f2d1d85a865846879791063b10cb9941
I20260806 08:57:14.405396 26432 ts_tablet_manager.cc:1939] T f2d1d85a865846879791063b10cb9941 P a655b21df3b54f899bc9eb9e148a4165: Deleting consensus metadata
I20260806 08:57:14.405341 25466 catalog_manager.cc:4841] TS 5c142761594e4d508e1d62647d7e84bb (127.22.98.1:33053): tablet f2d1d85a865846879791063b10cb9941 (table test-table [id=4bc48138b8ce45f9b081ab3770088048]) successfully deleted
I20260806 08:57:14.405917 25466 catalog_manager.cc:4841] TS 1b9db61f0c0c48f6b3bbbaf59c80c798 (127.22.98.2:46261): tablet f2d1d85a865846879791063b10cb9941 (table test-table [id=4bc48138b8ce45f9b081ab3770088048]) successfully deleted
I20260806 08:57:14.406409 25466 catalog_manager.cc:4841] TS a655b21df3b54f899bc9eb9e148a4165 (127.22.98.3:39053): tablet f2d1d85a865846879791063b10cb9941 (table test-table [id=4bc48138b8ce45f9b081ab3770088048]) successfully deleted
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:14.417132 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:14.391756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.22.98.1:33053)
0806 08:57:14.393870 (+ 2114us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:14.393875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:14.394510 (+ 635us) server_negotiation.cc:407] Connection header received
0806 08:57:14.394579 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:14.394584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:14.394960 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:14.395080 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:14.397014 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.398319 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.400340 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.401136 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.401266 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:14.410902 (+ 9636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:14.410919 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:14.410937 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:14.410978 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:14.414110 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.414774 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.414793 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.414797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.415031 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.415480 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.415483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.415486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.415948 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:14.416232 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:14.416571 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 08:57:14.416799 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1964,"thread_start_us":1832,"threads_started":1}
W20260806 08:57:14.417861 25794 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40520
W20260806 08:57:14.418522 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25591: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:14.418813 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 25591
I20260806 08:57:14.453644 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25807
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:14.469296 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:14.454447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.22.98.2:46261)
0806 08:57:14.455137 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:14.455141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:14.455159 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:57:14.455216 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:14.455220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:14.455278 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:14.455375 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:14.456723 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.457488 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.458732 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.459255 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.459355 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:14.463641 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:14.463656 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:14.463708 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:14.463751 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:14.466455 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.467155 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.467165 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.467167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.467306 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.467717 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.467721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.467723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.468112 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:14.468417 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:14.468726 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 08:57:14.468969 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":452,"threads_started":1}
W20260806 08:57:14.469942 25992 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39762
W20260806 08:57:14.470561 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25807: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:14.470769 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 25807
I20260806 08:57:14.498531 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26001
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[25422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:14.514075 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:14.499456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.22.98.3:39053)
0806 08:57:14.500071 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:14.500077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:14.500099 (+ 22us) server_negotiation.cc:407] Connection header received
0806 08:57:14.500157 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:14.500161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:14.500231 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:14.500329 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:14.501654 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.502443 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.503790 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:14.504233 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:14.504341 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:14.508571 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:14.508587 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:14.508607 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:14.508645 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:14.511262 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.512001 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.512009 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.512011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.512140 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:14.512513 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:14.512516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:14.512518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:14.512898 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:14.513177 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:14.513492 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 08:57:14.513741 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":363,"threads_started":1}
W20260806 08:57:14.514714 26189 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34092
W20260806 08:57:14.515329 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:14.515493 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 26001
I20260806 08:57:14.540652 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 25442
I20260806 08:57:14.541837 25548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:14.659847 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 25442
2026-08-06T08:57:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9970 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[26444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[26444](info): set up 2 sockets
Aug 06 08:57:14 dist-test-slave-jpj1 krb5kdc[26444](info): commencing operation
krb5kdc: starting...
W20260806 08:57:15.839818 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:15 dist-test-slave-jpj1 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:16.063503 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36269
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:36269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:16.401190 26460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:16.401494 26460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:16.401631 26460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:16.408906 26460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:16.409022 26460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:16.409092 26460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:16.409169 26460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:16.409215 26460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:16.419247 26460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:36269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:16.420612 26460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:16.422065 26460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:16.434074 26466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:16.434146 26467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:16.434105 26469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:16.434405 26460 server_base.cc:1034] running on GCE node
I20260806 08:57:16.435495 26460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:16.436967 26460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:16.438207 26460 hybrid_clock.cc:648] HybridClock initialized: now 1786006636438138 us; error 33 us; skew 500 ppm
I20260806 08:57:16.438572 26460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:16 dist-test-slave-jpj1 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006636, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:16.441676 26460 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:16.443132 26460 webserver.cc:460] Webserver started at http://127.22.98.62:42335/ using document root <none> and password file <none>
I20260806 08:57:16.443881 26460 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:16.443989 26460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:16.444309 26460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:16.447301 26460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "3140b6b87de6450abfe21c6bede83a56"
format_stamp: "Formatted at 2026-08-06 08:57:16 on dist-test-slave-jpj1"
server_key: "927ea4fa87d370e54603f5bac396fb6f"
server_key_iv: "e88f8b677cf81b92867165c9e718c1e6"
server_key_version: "encryptionkey@0"
I20260806 08:57:16.448097 26460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "3140b6b87de6450abfe21c6bede83a56"
format_stamp: "Formatted at 2026-08-06 08:57:16 on dist-test-slave-jpj1"
server_key: "927ea4fa87d370e54603f5bac396fb6f"
server_key_iv: "e88f8b677cf81b92867165c9e718c1e6"
server_key_version: "encryptionkey@0"
I20260806 08:57:16.453378 26460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:57:16.456900 26481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:16.458200 26460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:16.458359 26460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "3140b6b87de6450abfe21c6bede83a56"
format_stamp: "Formatted at 2026-08-06 08:57:16 on dist-test-slave-jpj1"
server_key: "927ea4fa87d370e54603f5bac396fb6f"
server_key_iv: "e88f8b677cf81b92867165c9e718c1e6"
server_key_version: "encryptionkey@0"
I20260806 08:57:16.458551 26460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:16.494513 26460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:16.498694 26460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:16.498966 26460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:16.516355 26460 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:36269
I20260806 08:57:16.516387 26557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:36269 every 8 connection(s)
I20260806 08:57:16.517803 26460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:16.521672 26559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:16.525676 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 26460
I20260806 08:57:16.525924 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:16.526691 22920 external_mini_cluster.cc:1428] Setting key b8548ed0adf95acf6c29df90e9bcd145
I20260806 08:57:16.533429 26559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56: Bootstrap starting.
I20260806 08:57:16.537184 26559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:16.538553 26559 log.cc:824] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:16.542411 26559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56: No bootstrap required, opened a new log
Aug 06 08:57:16 dist-test-slave-jpj1 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:16.551126 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:16.528826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.22.98.62:36269)
0806 08:57:16.529606 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:16.529618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:16.529666 (+ 48us) server_negotiation.cc:407] Connection header received
0806 08:57:16.531946 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:16.531972 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:16.532364 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:16.532686 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:16.534513 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:16.535998 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:16.537762 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:16.538825 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:16.539016 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:16.543765 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:16.543774 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:16.543790 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:16.543831 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:16.546970 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:16.547811 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:16.547818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:16.547829 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:16.548001 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:16.548376 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:16.548379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:16.548381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:16.548944 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:16.549184 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:16.549649 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 08:57:16.550048 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":348,"threads_started":1}
I20260806 08:57:16.551331 26559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } }
I20260806 08:57:16.551759 26559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:16.551869 26559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3140b6b87de6450abfe21c6bede83a56, State: Initialized, Role: FOLLOWER
I20260806 08:57:16.552449 26559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } }
I20260806 08:57:16.552744 26559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:16.552887 26559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:16.553071 26559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:16.555627 26559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } }
I20260806 08:57:16.556073 26559 leader_election.cc:302] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3140b6b87de6450abfe21c6bede83a56; no voters:
I20260806 08:57:16.556581 26559 leader_election.cc:288] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:16.556715 26567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:16.557689 26567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [term 1 LEADER]: Becoming Leader. State: Replica: 3140b6b87de6450abfe21c6bede83a56, State: Running, Role: LEADER
I20260806 08:57:16.558244 26567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } }
I20260806 08:57:16.558626 26559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:16.560806 26573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3140b6b87de6450abfe21c6bede83a56. Latest consensus state: current_term: 1 leader_uuid: "3140b6b87de6450abfe21c6bede83a56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } } }
I20260806 08:57:16.561120 26573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:16.561651 26568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3140b6b87de6450abfe21c6bede83a56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140b6b87de6450abfe21c6bede83a56" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36269 } } }
I20260806 08:57:16.561956 26568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:16.562184 26581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:16.576248 26581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:16.585649 26581 catalog_manager.cc:1347] Generated new cluster ID: 2d4c9cdec3fa4486be9f032c85e019bf
I20260806 08:57:16.585777 26581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:16.630715 26581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:16.632697 26581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:16.650194 26581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3140b6b87de6450abfe21c6bede83a56: Generated new TSK 0
I20260806 08:57:16.651355 26581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:16.712862 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:17.053023 26601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:17.053323 26601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:17.053418 26601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:17.060766 26601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:17.060896 26601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:17.061020 26601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:17.071244 26601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:17.072750 26601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:17.074236 26601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:17.088092 26607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:17.088135 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:17.088161 26608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:17.089102 26601 server_base.cc:1034] running on GCE node
I20260806 08:57:17.089766 26601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:17.090848 26601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:17.092077 26601 hybrid_clock.cc:648] HybridClock initialized: now 1786006637092010 us; error 55 us; skew 500 ppm
I20260806 08:57:17.092446 26601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:17 dist-test-slave-jpj1 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006637, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:17.095976 26601 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:17.097788 26601 webserver.cc:460] Webserver started at http://127.22.98.1:40321/ using document root <none> and password file <none>
I20260806 08:57:17.098551 26601 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:17.098685 26601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:17.099018 26601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:17.102350 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "6b515163a61144ec8c4bff8d0a622f6e"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "98149a3cb6b2b7465f9122677f4a52c6"
server_key_iv: "b77a4abb8ef1b2b4a502449759da2556"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.103178 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "6b515163a61144ec8c4bff8d0a622f6e"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "98149a3cb6b2b7465f9122677f4a52c6"
server_key_iv: "b77a4abb8ef1b2b4a502449759da2556"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.109018 26601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 08:57:17.112884 26623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.114349 26601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:57:17.114523 26601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "6b515163a61144ec8c4bff8d0a622f6e"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "98149a3cb6b2b7465f9122677f4a52c6"
server_key_iv: "b77a4abb8ef1b2b4a502449759da2556"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.114725 26601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:17.127499 26601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:17.132167 26601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:17.132498 26601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:17.133553 26601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:17.135038 26601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:17.135123 26601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.135254 26601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:17.135308 26601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.167050 26601 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:34257
I20260806 08:57:17.167100 26779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:34257 every 8 connection(s)
I20260806 08:57:17.168512 26601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:17.175016 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 26601
I20260806 08:57:17.175280 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:17.176128 22920 external_mini_cluster.cc:1428] Setting key b23eb0169c989d6c75bb084d556078ec
Aug 06 08:57:17 dist-test-slave-jpj1 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006637, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:17.192183 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:17.171355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:37037 (local address 127.22.98.62:36269)
0806 08:57:17.171831 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:17.171836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:17.172799 (+ 963us) server_negotiation.cc:407] Connection header received
0806 08:57:17.174860 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:17.174865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:17.174942 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:17.175042 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:17.177187 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.177936 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.180193 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.180633 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.180735 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:17.185329 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:17.185335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:17.185339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:17.185378 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:17.188560 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:17.189432 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:17.189438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:17.189442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:17.189618 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:17.190169 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:17.190175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:17.190179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:17.190575 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:17.190729 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:17.191530 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 08:57:17.191828 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":227,"threads_started":1}
I20260806 08:57:17.193156 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:17.171845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36269 (local address 127.22.98.1:37037)
0806 08:57:17.172623 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:17.172665 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:17.174436 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:17.175363 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:17.175376 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:17.176195 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:17.176994 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:17.177017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.178082 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.178086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:17.180033 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:17.180050 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.180926 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.180935 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:17.181239 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:17.182008 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:17.182039 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:17.184976 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:17.188736 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:17.188762 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:17.188777 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:17.189265 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:17.189775 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:17.189779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:17.189781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:17.190018 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:17.190735 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:17.190742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:17.191111 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 08:57:17.191373 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 08:57:17.191914 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":285,"threads_started":1}
I20260806 08:57:17.194908 26780 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36269
I20260806 08:57:17.195338 26780 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:17.196353 26780 heartbeater.cc:511] Master 127.22.98.62:36269 requested a full tablet report, sending...
I20260806 08:57:17.198928 26505 ts_manager.cc:194] Registered new tserver with Master: 6b515163a61144ec8c4bff8d0a622f6e (127.22.98.1:34257)
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
I20260806 08:57:17.200567 26505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:37037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:17.232570 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:17.578696 26787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:17.579013 26787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:17.579172 26787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:17.586454 26787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:17.586584 26787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:17.586750 26787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:17.596765 26787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:17.598125 26787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:17.599463 26787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:17.612193 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:17.612397 26794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:17.612272 26799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:17.613418 26787 server_base.cc:1034] running on GCE node
I20260806 08:57:17.614054 26787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:17.614971 26787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:17.616181 26787 hybrid_clock.cc:648] HybridClock initialized: now 1786006637616115 us; error 52 us; skew 500 ppm
I20260806 08:57:17.616488 26787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:17 dist-test-slave-jpj1 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006637, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:17.619765 26787 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:17.621371 26787 webserver.cc:460] Webserver started at http://127.22.98.2:44991/ using document root <none> and password file <none>
I20260806 08:57:17.622054 26787 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:17.622159 26787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:17.622462 26787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:17.625990 26787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "6c7ed7d185f24f46901e9f6b509b336c"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "dd853807464084352e9531166708e009"
server_key_iv: "fa58974b97ebe27851c8766c7dc09416"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.627061 26787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "6c7ed7d185f24f46901e9f6b509b336c"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "dd853807464084352e9531166708e009"
server_key_iv: "fa58974b97ebe27851c8766c7dc09416"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.634559 26787 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 08:57:17.639642 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.641562 26787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 08:57:17.641749 26787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "6c7ed7d185f24f46901e9f6b509b336c"
format_stamp: "Formatted at 2026-08-06 08:57:17 on dist-test-slave-jpj1"
server_key: "dd853807464084352e9531166708e009"
server_key_iv: "fa58974b97ebe27851c8766c7dc09416"
server_key_version: "encryptionkey@0"
I20260806 08:57:17.642017 26787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:17.666798 26787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:17.672626 26787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:17.672972 26787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:17.674007 26787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:17.675421 26787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:17.675498 26787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.675622 26787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:17.675678 26787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:17.706881 26787 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46749
I20260806 08:57:17.706924 26988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46749 every 8 connection(s)
I20260806 08:57:17.708251 26787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:17.715127 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 26787
I20260806 08:57:17.715296 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:17.715770 22920 external_mini_cluster.cc:1428] Setting key f7af122d6c6aae1f04bf1b3c4d22ca23
Aug 06 08:57:17 dist-test-slave-jpj1 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006637, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:17.728086 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:17.710908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:45127 (local address 127.22.98.62:36269)
0806 08:57:17.711392 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:17.711398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:17.712396 (+ 998us) server_negotiation.cc:407] Connection header received
0806 08:57:17.714270 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:17.714282 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:17.714358 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:17.714453 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:17.716112 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.717046 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.718712 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.719180 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.719299 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:17.722713 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:17.722718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:17.722722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:17.722752 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:17.725147 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:17.725839 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:17.725845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:17.725847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:17.725978 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:17.726381 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:17.726385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:17.726387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:17.726644 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:17.726772 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:17.727466 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 08:57:17.727730 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":218,"threads_started":1}
I20260806 08:57:17.729063 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:17.711459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36269 (local address 127.22.98.2:45127)
0806 08:57:17.712216 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:17.712260 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:17.713977 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:17.714649 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:17.714657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:17.715200 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:17.715932 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:17.715953 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.717197 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.717201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:17.718561 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:17.718574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:17.719407 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:17.719414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:17.719686 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:17.720348 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:17.720387 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:17.722498 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:17.725275 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:17.725285 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:17.725305 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:17.725715 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:17.726081 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:17.726086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:17.726090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:17.726269 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:17.726765 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:17.726773 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:17.727104 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 08:57:17.727346 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 08:57:17.727931 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":274,"threads_started":1}
I20260806 08:57:17.730528 26990 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36269
I20260806 08:57:17.730829 26990 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:17.731601 26990 heartbeater.cc:511] Master 127.22.98.62:36269 requested a full tablet report, sending...
I20260806 08:57:17.733136 26505 ts_manager.cc:194] Registered new tserver with Master: 6c7ed7d185f24f46901e9f6b509b336c (127.22.98.2:46749)
I20260806 08:57:17.733959 26505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:45127
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:17.767655 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:18.107627 27000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:18.107954 27000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:18.108050 27000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:18.115427 27000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:18.115566 27000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:18.115682 27000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:18.125746 27000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:36269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:18.127089 27000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:18.128453 27000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:18.142732 27007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:18.142628 27010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:18.142628 27008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:18.143680 27000 server_base.cc:1034] running on GCE node
I20260806 08:57:18.144388 27000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:18.145512 27000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:18.146746 27000 hybrid_clock.cc:648] HybridClock initialized: now 1786006638146672 us; error 58 us; skew 500 ppm
I20260806 08:57:18.147109 27000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:18 dist-test-slave-jpj1 krb5kdc[26444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006638, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:18.150473 27000 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:18.152189 27000 webserver.cc:460] Webserver started at http://127.22.98.3:39077/ using document root <none> and password file <none>
I20260806 08:57:18.152943 27000 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:18.153079 27000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:18.153404 27000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:18.156867 27000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "3e45c2b57cff4f68a84bd54feda2ceaa"
format_stamp: "Formatted at 2026-08-06 08:57:18 on dist-test-slave-jpj1"
server_key: "d5195a6798d7f4f5bbc6ae0916f61f87"
server_key_iv: "1a0aaec67da38ab6e8cc24a6db120dc3"
server_key_version: "encryptionkey@0"
I20260806 08:57:18.157922 27000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "3e45c2b57cff4f68a84bd54feda2ceaa"
format_stamp: "Formatted at 2026-08-06 08:57:18 on dist-test-slave-jpj1"
server_key: "d5195a6798d7f4f5bbc6ae0916f61f87"
server_key_iv: "1a0aaec67da38ab6e8cc24a6db120dc3"
server_key_version: "encryptionkey@0"
I20260806 08:57:18.164112 27000 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 08:57:18.168140 27022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:18.169622 27000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:57:18.169790 27000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "3e45c2b57cff4f68a84bd54feda2ceaa"
format_stamp: "Formatted at 2026-08-06 08:57:18 on dist-test-slave-jpj1"
server_key: "d5195a6798d7f4f5bbc6ae0916f61f87"
server_key_iv: "1a0aaec67da38ab6e8cc24a6db120dc3"
server_key_version: "encryptionkey@0"
I20260806 08:57:18.169987 27000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:18.193509 27000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:18.198046 27000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:18.198343 27000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:18.199406 27000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:18.200867 27000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:18.200959 27000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:18.201102 27000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:18.201154 27000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:18.204262 26780 heartbeater.cc:503] Master 127.22.98.62:36269 was elected leader, sending a full tablet report...
I20260806 08:57:18.234843 27000 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:36381
I20260806 08:57:18.234933 27180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:36381 every 8 connection(s)
I20260806 08:57:18.236259 27000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:18.242082 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27000
I20260806 08:57:18.242285 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:18.242722 22920 external_mini_cluster.cc:1428] Setting key ff33704db2fddedf91ec84233cdc35ad
Aug 06 08:57:18 dist-test-slave-jpj1 krb5kdc[26444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006638, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:18.258425 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:18.238988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:51215 (local address 127.22.98.62:36269)
0806 08:57:18.239470 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:18.239475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:18.240602 (+ 1127us) server_negotiation.cc:407] Connection header received
0806 08:57:18.242560 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:18.242565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:18.242678 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:18.242776 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:18.244657 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:18.245608 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:18.247301 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:18.247806 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:18.247918 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:18.251809 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:18.251815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:18.251818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:18.251859 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:18.254556 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:18.255516 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:18.255520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:18.255523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:18.255730 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:18.256270 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:18.256274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:18.256289 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:18.256589 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:18.256741 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:18.257776 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0806 08:57:18.258097 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":267,"threads_started":1}
I20260806 08:57:18.259493 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:18.239586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:36269 (local address 127.22.98.3:51215)
0806 08:57:18.240406 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:18.240456 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:18.242245 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:18.243054 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:18.243063 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:18.243776 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:18.244468 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:18.244492 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:18.245782 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:18.245788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:18.247111 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:18.247126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:18.248054 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:18.248060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:18.248294 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:18.248972 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:18.248997 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:18.251452 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:18.254824 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:18.254834 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:18.254851 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:18.255353 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:18.255894 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:18.255898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:18.255900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:18.256116 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:18.256837 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:18.256845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:18.257356 (+ 511us) client_negotiation.cc:769] Sending connection context
0806 08:57:18.257625 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 08:57:18.258167 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":297,"threads_started":1}
I20260806 08:57:18.260947 27182 heartbeater.cc:347] Connected to a master server at 127.22.98.62:36269
I20260806 08:57:18.261251 27182 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:18.262049 27182 heartbeater.cc:511] Master 127.22.98.62:36269 requested a full tablet report, sending...
I20260806 08:57:18.263523 26505 ts_manager.cc:194] Registered new tserver with Master: 3e45c2b57cff4f68a84bd54feda2ceaa (127.22.98.3:36381)
I20260806 08:57:18.264389 26505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:51215
I20260806 08:57:18.271413 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 08:57:18.309782 27185 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:18.307057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.22.98.62:36269)
0806 08:57:18.307328 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:18.307335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:18.307699 (+ 364us) server_negotiation.cc:407] Connection header received
0806 08:57:18.307916 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:18.307921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:18.308013 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:18.308109 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:18.309500 (+ 1391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42922: BlockingRecv error: recv got EOF from 127.0.0.1:42922 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 08:57:18.313002 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26601
W20260806 08:57:18.315893 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26601: Not authorized: Client connection negotiation failed: client connection to 127.22.98.1:34257: server requires authentication, but client does not have Kerberos credentials available
I20260806 08:57:18.316087 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 26601
W20260806 08:57:18.316141 27200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:18.313988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.22.98.1:34257)
0806 08:57:18.314567 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:18.314574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:18.314602 (+ 28us) server_negotiation.cc:407] Connection header received
0806 08:57:18.314669 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:18.314675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:18.314883 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:18.315016 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:18.315797 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39978: BlockingRecv error: recv got EOF from 127.0.0.1:39978 (error 108)
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":264,"threads_started":1}
I20260806 08:57:18.339468 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26787
W20260806 08:57:18.342268 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26787: Not authorized: Client connection negotiation failed: client connection to 127.22.98.2:46749: server requires authentication, but client does not have Kerberos credentials available
I20260806 08:57:18.342434 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 26787
I20260806 08:57:18.364763 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27000
W20260806 08:57:18.367853 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27000: Not authorized: Client connection negotiation failed: client connection to 127.22.98.3:36381: server requires authentication, but client does not have Kerberos credentials available
I20260806 08:57:18.367985 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27000
I20260806 08:57:18.387938 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 26460
I20260806 08:57:18.389262 26550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:18.491967 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 26460
2026-08-06T08:57:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3829 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:18 dist-test-slave-jpj1 krb5kdc[27210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:18 dist-test-slave-jpj1 krb5kdc[27210](info): set up 2 sockets
Aug 06 08:57:18 dist-test-slave-jpj1 krb5kdc[27210](info): commencing operation
krb5kdc: starting...
W20260806 08:57:19.662237 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:19 dist-test-slave-jpj1 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:19.891438 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34539
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:39383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:20.226949 27231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:20.227265 27231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:20.227380 27231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:20.234814 27231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:20.234941 27231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:20.235002 27231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:20.235086 27231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:20.235155 27231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:20.245452 27231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:20.246819 27231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:20.248231 27231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:20.260102 27239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:20.260324 27240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:20.260666 27242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:20.261363 27231 server_base.cc:1034] running on GCE node
I20260806 08:57:20.262220 27231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:20.263756 27231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:20.264999 27231 hybrid_clock.cc:648] HybridClock initialized: now 1786006640264944 us; error 43 us; skew 500 ppm
I20260806 08:57:20.265393 27231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:20 dist-test-slave-jpj1 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006640, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:20.271947 27231 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:20.273653 27231 webserver.cc:460] Webserver started at http://127.22.98.62:32919/ using document root <none> and password file <none>
I20260806 08:57:20.274389 27231 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:20.274513 27231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:20.274830 27231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:20.278038 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "29aafeedf6ce4e4185a529c360d2d966"
format_stamp: "Formatted at 2026-08-06 08:57:20 on dist-test-slave-jpj1"
server_key: "2378bc39708c852539d8a474dd1859f7"
server_key_iv: "774cdcc343310c41a549ceb659fb85c4"
server_key_version: "encryptionkey@0"
I20260806 08:57:20.278853 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "29aafeedf6ce4e4185a529c360d2d966"
format_stamp: "Formatted at 2026-08-06 08:57:20 on dist-test-slave-jpj1"
server_key: "2378bc39708c852539d8a474dd1859f7"
server_key_iv: "774cdcc343310c41a549ceb659fb85c4"
server_key_version: "encryptionkey@0"
I20260806 08:57:20.284246 27231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 08:57:20.287868 27251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:20.289211 27231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:20.289393 27231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "29aafeedf6ce4e4185a529c360d2d966"
format_stamp: "Formatted at 2026-08-06 08:57:20 on dist-test-slave-jpj1"
server_key: "2378bc39708c852539d8a474dd1859f7"
server_key_iv: "774cdcc343310c41a549ceb659fb85c4"
server_key_version: "encryptionkey@0"
I20260806 08:57:20.289593 27231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:20.307694 27231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:20.312070 27231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:20.312342 27231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:20.330869 27231 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34539
I20260806 08:57:20.330902 27322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34539 every 8 connection(s)
I20260806 08:57:20.332386 27231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:20.333451 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27231
I20260806 08:57:20.333752 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:20.334383 22920 external_mini_cluster.cc:1428] Setting key 095296135aa6af0f13f28e5ef73273dd
I20260806 08:57:20.337005 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 08:57:20 dist-test-slave-jpj1 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:20.348374 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966: Bootstrap starting.
I20260806 08:57:20.351980 27323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:20.353078 27323 log.cc:824] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:20.356379 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966: No bootstrap required, opened a new log
I20260806 08:57:20.357019 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:20.336276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.22.98.62:34539)
0806 08:57:20.337200 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:20.337216 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:20.337268 (+ 52us) server_negotiation.cc:407] Connection header received
0806 08:57:20.339023 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:20.339049 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:20.339482 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:20.339948 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:20.341578 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:20.343080 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:20.344397 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:20.345127 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:20.345342 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:20.349366 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:20.349375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:20.349394 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:20.349435 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:20.352713 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:20.353434 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:20.353442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:20.353449 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:20.353685 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:20.354142 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:20.354148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:20.354151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:20.354760 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:20.354979 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:20.355508 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 08:57:20.355918 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":395,"threads_started":1}
I20260806 08:57:20.361099 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 08:57:20.366294 27323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } }
I20260806 08:57:20.366768 27323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:20.366869 27323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29aafeedf6ce4e4185a529c360d2d966, State: Initialized, Role: FOLLOWER
I20260806 08:57:20.367554 27323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } }
I20260806 08:57:20.367828 27323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:20.367959 27323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:20.368129 27323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:20.370975 27323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } }
I20260806 08:57:20.371502 27323 leader_election.cc:302] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29aafeedf6ce4e4185a529c360d2d966; no voters:
I20260806 08:57:20.372148 27323 leader_election.cc:288] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:20.372308 27335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:20.374182 27323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:20.379161 27335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [term 1 LEADER]: Becoming Leader. State: Replica: 29aafeedf6ce4e4185a529c360d2d966, State: Running, Role: LEADER
I20260806 08:57:20.379897 27335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } }
I20260806 08:57:20.382866 27336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29aafeedf6ce4e4185a529c360d2d966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } } }
I20260806 08:57:20.383625 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29aafeedf6ce4e4185a529c360d2d966. Latest consensus state: current_term: 1 leader_uuid: "29aafeedf6ce4e4185a529c360d2d966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29aafeedf6ce4e4185a529c360d2d966" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34539 } } }
I20260806 08:57:20.383275 27336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:20.383980 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:20.391988 27351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:20.397783 27351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:20.418773 27351 catalog_manager.cc:1347] Generated new cluster ID: fc21eaf778984fa7aafab81095fd688f
I20260806 08:57:20.418918 27351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:20.433769 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:20.423909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52888 (local address 127.22.98.62:34539)
0806 08:57:20.424130 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:20.424137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:20.424862 (+ 725us) server_negotiation.cc:407] Connection header received
0806 08:57:20.425063 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:20.425068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:20.425153 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:20.425271 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:20.426706 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:20.427854 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:20.429773 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:20.430407 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:20.430514 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:20.431815 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:20.431822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:20.431827 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:20.432890 (+ 1063us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:20.432963 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:20.432994 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:20.433106 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:20.433261 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 08:57:20.433363 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 08:57:20.441159 27351 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:20.442703 27351 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:20.450585 27351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29aafeedf6ce4e4185a529c360d2d966: Generated new TSK 0
I20260806 08:57:20.451714 27351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:57:20.464396 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27231
I20260806 08:57:20.465548 27317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:20.558241 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27231
2026-08-06T08:57:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2061 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:20 dist-test-slave-jpj1 krb5kdc[27375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:20 dist-test-slave-jpj1 krb5kdc[27375](info): set up 2 sockets
Aug 06 08:57:20 dist-test-slave-jpj1 krb5kdc[27375](info): commencing operation
krb5kdc: starting...
W20260806 08:57:21.728713 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:21 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:21.951614 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43399
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:43399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:22.292357 27394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:22.292712 27394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:22.292843 27394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:22.300419 27394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:22.300551 27394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:22.300627 27394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:22.300704 27394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:22.300765 27394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:22.311080 27394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:43399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:22.312564 27394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:22.313999 27394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:22.326051 27413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:22.326077 27407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:22.326752 27406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:22.327124 27394 server_base.cc:1034] running on GCE node
I20260806 08:57:22.328011 27394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:22.329490 27394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:22.330714 27394 hybrid_clock.cc:648] HybridClock initialized: now 1786006642330637 us; error 66 us; skew 500 ppm
I20260806 08:57:22.331071 27394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:22 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:22.334420 27394 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:22.336015 27394 webserver.cc:460] Webserver started at http://127.22.98.62:42245/ using document root <none> and password file <none>
I20260806 08:57:22.336731 27394 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:22.336854 27394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:22.337167 27394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:22.340302 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "b66706eb21c14c09a507597988412af6"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "e68236aee226eba864b46e435cd16a01"
server_key_iv: "3a056d48fe753313eb65498245592077"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.341094 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "b66706eb21c14c09a507597988412af6"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "e68236aee226eba864b46e435cd16a01"
server_key_iv: "3a056d48fe753313eb65498245592077"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.346714 27394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:22.350651 27423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:22.352315 27394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:22.352516 27394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "b66706eb21c14c09a507597988412af6"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "e68236aee226eba864b46e435cd16a01"
server_key_iv: "3a056d48fe753313eb65498245592077"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.352723 27394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:22.369684 27394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:22.374267 27394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:22.374571 27394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:22.393637 27394 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:43399
I20260806 08:57:22.393659 27495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:43399 every 8 connection(s)
I20260806 08:57:22.395097 27394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:22.399842 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:22.404187 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27394
I20260806 08:57:22.404402 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:22.405061 22920 external_mini_cluster.cc:1428] Setting key cca81c84c80cc1824e9e446976fb402b
I20260806 08:57:22.413415 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6: Bootstrap starting.
I20260806 08:57:22.417518 27497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:22.419047 27497 log.cc:824] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6: Log is configured to *not* fsync() on all Append() calls
Aug 06 08:57:22 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:22.422590 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6: No bootstrap required, opened a new log
I20260806 08:57:22.433557 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:22.407431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.22.98.62:43399)
0806 08:57:22.408373 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:22.408395 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:22.408460 (+ 65us) server_negotiation.cc:407] Connection header received
0806 08:57:22.411351 (+ 2891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:22.411395 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:22.412061 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:22.412601 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:22.414054 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:22.416148 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:22.417629 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:22.418737 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:22.418952 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:22.423657 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:22.423670 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:22.423704 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:22.423763 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:22.427994 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:22.428886 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:22.428897 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:22.428912 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:22.429177 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:22.429698 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:22.429703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:22.429707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:22.430579 (+ 872us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:22.430948 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:22.431437 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 08:57:22.432006 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":376,"threads_started":1}
I20260806 08:57:22.433596 27497 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } }
I20260806 08:57:22.434103 27497 raft_consensus.cc:374] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:22.434229 27497 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66706eb21c14c09a507597988412af6, State: Initialized, Role: FOLLOWER
I20260806 08:57:22.434860 27497 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } }
I20260806 08:57:22.435081 27497 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:22.435286 27497 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:22.435473 27497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:22.437944 27497 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } }
I20260806 08:57:22.438465 27497 leader_election.cc:302] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66706eb21c14c09a507597988412af6; no voters:
I20260806 08:57:22.439139 27497 leader_election.cc:288] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:22.439244 27504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:22.439954 27504 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [term 1 LEADER]: Becoming Leader. State: Replica: b66706eb21c14c09a507597988412af6, State: Running, Role: LEADER
I20260806 08:57:22.440538 27504 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } }
I20260806 08:57:22.441550 27497 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:22.443455 27510 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66706eb21c14c09a507597988412af6. Latest consensus state: current_term: 1 leader_uuid: "b66706eb21c14c09a507597988412af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } } }
I20260806 08:57:22.443878 27510 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:22.444590 27509 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b66706eb21c14c09a507597988412af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66706eb21c14c09a507597988412af6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43399 } } }
I20260806 08:57:22.444928 27509 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:22.446177 27514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:22.451704 27514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:22.462518 27514 catalog_manager.cc:1347] Generated new cluster ID: 17369f8110ae4f17bce0de22eb112ba9
I20260806 08:57:22.462648 27514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:22.477417 27514 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:22.479724 27514 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:22.502816 27514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b66706eb21c14c09a507597988412af6: Generated new TSK 0
I20260806 08:57:22.504046 27514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:22.563134 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:22.900028 27532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:22.900326 27532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:22.900460 27532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:22.907707 27532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:22.907831 27532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:22.907979 27532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:22.917956 27532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:22.919294 27532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:22.920662 27532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:22.934278 27540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:22.934535 27543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:22.934679 27539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:22.935397 27532 server_base.cc:1034] running on GCE node
I20260806 08:57:22.936146 27532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:22.937211 27532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:22.938464 27532 hybrid_clock.cc:648] HybridClock initialized: now 1786006642938394 us; error 57 us; skew 500 ppm
I20260806 08:57:22.938791 27532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:22 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:22.942332 27532 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:22.944028 27532 webserver.cc:460] Webserver started at http://127.22.98.1:46253/ using document root <none> and password file <none>
I20260806 08:57:22.944751 27532 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:22.944876 27532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:22.945221 27532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:22.948491 27532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "26c030a465c84ba7948b00a66e5ddd1d"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "4f34c166ae6036ed8f17572fe907befd"
server_key_iv: "8e151e993adfad8b46b6784674321244"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.949335 27532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "26c030a465c84ba7948b00a66e5ddd1d"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "4f34c166ae6036ed8f17572fe907befd"
server_key_iv: "8e151e993adfad8b46b6784674321244"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.954995 27532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:22.959120 27552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:22.960719 27532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:22.960917 27532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "26c030a465c84ba7948b00a66e5ddd1d"
format_stamp: "Formatted at 2026-08-06 08:57:22 on dist-test-slave-jpj1"
server_key: "4f34c166ae6036ed8f17572fe907befd"
server_key_iv: "8e151e993adfad8b46b6784674321244"
server_key_version: "encryptionkey@0"
I20260806 08:57:22.961135 27532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:22.976673 27532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:22.981007 27532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:22.981323 27532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:22.982344 27532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:22.983781 27532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:22.983865 27532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:22.983999 27532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:22.984056 27532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:23.016685 27532 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:35493
I20260806 08:57:23.016791 27732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:35493 every 8 connection(s)
I20260806 08:57:23.018044 27532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:23.027426 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27532
I20260806 08:57:23.027683 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:23.028470 22920 external_mini_cluster.cc:1428] Setting key 651eeb4c844a1cc7a53d7d05c32d94d7
Aug 06 08:57:23 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:23.038909 27737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:23.020617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:46351 (local address 127.22.98.62:43399)
0806 08:57:23.021056 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:23.021060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:23.021970 (+ 910us) server_negotiation.cc:407] Connection header received
0806 08:57:23.023833 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:23.023839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:23.023920 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:23.024018 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:23.025597 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.026337 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.028492 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.028934 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.029066 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:23.033094 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:23.033100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:23.033103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:23.033137 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:23.035694 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:23.036493 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:23.036498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:23.036500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:23.036631 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:23.037071 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:23.037075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:23.037077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:23.037362 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:23.037510 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:23.038260 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 08:57:23.038513 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":227,"threads_started":1}
I20260806 08:57:23.040016 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:23.021061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43399 (local address 127.22.98.1:46351)
0806 08:57:23.021814 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:23.021853 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:23.023502 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:23.024291 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:23.024302 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:23.024818 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:23.025418 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:23.025436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.026492 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.026496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:23.028299 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:23.028317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.029183 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.029191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:23.029452 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:23.030223 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:23.030268 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:23.032836 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:23.035867 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:23.035877 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:23.035893 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:23.036358 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:23.036736 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:23.036741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:23.036745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:23.036950 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:23.037492 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:23.037501 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:23.037871 (+ 370us) client_negotiation.cc:769] Sending connection context
0806 08:57:23.038136 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 08:57:23.038732 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":284,"threads_started":1}
I20260806 08:57:23.041782 27733 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43399
I20260806 08:57:23.042125 27733 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:23.042930 27733 heartbeater.cc:511] Master 127.22.98.62:43399 requested a full tablet report, sending...
I20260806 08:57:23.045471 27442 ts_manager.cc:194] Registered new tserver with Master: 26c030a465c84ba7948b00a66e5ddd1d (127.22.98.1:35493)
I20260806 08:57:23.047703 27442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:46351
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:23.079778 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:23.422412 27742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:23.422714 27742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:23.422812 27742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:23.430212 27742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:23.430343 27742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:23.430469 27742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:23.440599 27742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:23.442334 27742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:23.443831 27742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:23.457037 27750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:23.457207 27751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:23.457365 27753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:23.458458 27742 server_base.cc:1034] running on GCE node
I20260806 08:57:23.459060 27742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:23.460065 27742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:23.461275 27742 hybrid_clock.cc:648] HybridClock initialized: now 1786006643461223 us; error 41 us; skew 500 ppm
I20260806 08:57:23.461584 27742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:23 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006643, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:23.464704 27742 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:23.466084 27742 webserver.cc:460] Webserver started at http://127.22.98.2:42039/ using document root <none> and password file <none>
I20260806 08:57:23.466763 27742 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:23.466866 27742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:23.467128 27742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:23.470021 27742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "27220a8a499b48c69fc1bdd623da3ebd"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "f9bbff257752b74d37d4f0d5f27a34fb"
server_key_iv: "8cbad8c078b4dd1a218938acfbe37eb5"
server_key_version: "encryptionkey@0"
I20260806 08:57:23.470774 27742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "27220a8a499b48c69fc1bdd623da3ebd"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "f9bbff257752b74d37d4f0d5f27a34fb"
server_key_iv: "8cbad8c078b4dd1a218938acfbe37eb5"
server_key_version: "encryptionkey@0"
I20260806 08:57:23.476169 27742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:23.479876 27765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:23.481382 27742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:23.481578 27742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "27220a8a499b48c69fc1bdd623da3ebd"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "f9bbff257752b74d37d4f0d5f27a34fb"
server_key_iv: "8cbad8c078b4dd1a218938acfbe37eb5"
server_key_version: "encryptionkey@0"
I20260806 08:57:23.481775 27742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:23.504346 27742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:23.508719 27742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:23.508999 27742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:23.510030 27742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:23.511409 27742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:23.511492 27742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:23.511639 27742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:23.511695 27742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:23.542829 27742 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:43575
I20260806 08:57:23.542876 27928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:43575 every 8 connection(s)
I20260806 08:57:23.544166 27742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:23.553059 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27742
I20260806 08:57:23.553288 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:23.553860 22920 external_mini_cluster.cc:1428] Setting key d391d50f5d789d671dfedaffd8501ed1
Aug 06 08:57:23 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006643, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:23.575485 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:23.546863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:47369 (local address 127.22.98.62:43399)
0806 08:57:23.547334 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:23.547338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:23.548518 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 08:57:23.550392 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:23.550401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:23.550474 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:23.550586 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:23.552149 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.552906 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.554986 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.555466 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.555670 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:23.559507 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:23.559513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:23.559516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:23.559581 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:23.571791 (+ 12210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:23.572758 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:23.572764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:23.572768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:23.572939 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:23.573474 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:23.573478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:23.573482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:23.573848 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:23.574014 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:23.574828 (+ 814us) server_negotiation.cc:299] Negotiation successful
0806 08:57:23.575075 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260806 08:57:23.576351 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:23.547456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43399 (local address 127.22.98.2:47369)
0806 08:57:23.548342 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:23.548389 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:23.550108 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:23.550775 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:23.550783 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:23.551329 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:23.551967 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:23.551992 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.553082 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.553088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:23.554858 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:23.554871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:23.555775 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:23.555783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:23.556068 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:23.556761 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:23.556787 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:23.559231 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:23.572019 (+ 12788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:23.572030 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:23.572048 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:23.572587 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:23.573109 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:23.573114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:23.573116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:23.573346 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:23.573995 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:23.574002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:23.574425 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 08:57:23.574694 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 08:57:23.575149 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":344,"threads_started":1}
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
I20260806 08:57:23.577924 27929 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43399
I20260806 08:57:23.578240 27929 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:23.578980 27929 heartbeater.cc:511] Master 127.22.98.62:43399 requested a full tablet report, sending...
I20260806 08:57:23.580569 27442 ts_manager.cc:194] Registered new tserver with Master: 27220a8a499b48c69fc1bdd623da3ebd (127.22.98.2:43575)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
I20260806 08:57:23.581590 27442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:47369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:23.607079 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:23.945343 27937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:23.945662 27937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:23.945801 27937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:23.953261 27937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:23.953403 27937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:23.953563 27937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:23.963835 27937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:23.965279 27937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:23.966732 27937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:23.980198 27943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:23.980351 27944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:23.980383 27947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:23.981727 27937 server_base.cc:1034] running on GCE node
I20260806 08:57:23.982450 27937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:23.983466 27937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:23.984735 27937 hybrid_clock.cc:648] HybridClock initialized: now 1786006643984669 us; error 49 us; skew 500 ppm
I20260806 08:57:23.985095 27937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:23 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006643, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:23.988390 27937 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:23.989981 27937 webserver.cc:460] Webserver started at http://127.22.98.3:40963/ using document root <none> and password file <none>
I20260806 08:57:23.990690 27937 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:23.990803 27937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:23.991140 27937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:23.994263 27937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "6d7b0b69403443458f622534e92cda33"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "dc9bcad32409b8aad332cf231dd3c792"
server_key_iv: "6372da792ca9167fb5618b63264bfddf"
server_key_version: "encryptionkey@0"
I20260806 08:57:23.995106 27937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "6d7b0b69403443458f622534e92cda33"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "dc9bcad32409b8aad332cf231dd3c792"
server_key_iv: "6372da792ca9167fb5618b63264bfddf"
server_key_version: "encryptionkey@0"
I20260806 08:57:24.000861 27937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 08:57:24.004721 27956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:24.006305 27937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:57:24.006479 27937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "6d7b0b69403443458f622534e92cda33"
format_stamp: "Formatted at 2026-08-06 08:57:23 on dist-test-slave-jpj1"
server_key: "dc9bcad32409b8aad332cf231dd3c792"
server_key_iv: "6372da792ca9167fb5618b63264bfddf"
server_key_version: "encryptionkey@0"
I20260806 08:57:24.006673 27937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:24.027920 27937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:24.032188 27937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:24.032475 27937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:24.033557 27937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:24.034996 27937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:24.035070 27937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:24.035185 27937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:24.035243 27937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:24.051281 27733 heartbeater.cc:503] Master 127.22.98.62:43399 was elected leader, sending a full tablet report...
I20260806 08:57:24.068070 27937 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:39449
I20260806 08:57:24.068120 28108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:39449 every 8 connection(s)
I20260806 08:57:24.069500 27937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:24.070013 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 27937
I20260806 08:57:24.070191 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:24.070680 22920 external_mini_cluster.cc:1428] Setting key f6b1e0f90e239280f918e50937f9edb8
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006643, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:24.091045 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.072555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:60379 (local address 127.22.98.62:43399)
0806 08:57:24.072784 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:24.072789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:24.074242 (+ 1453us) server_negotiation.cc:407] Connection header received
0806 08:57:24.076123 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:24.076129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:24.076204 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:24.076286 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:24.077851 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.078592 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.080235 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.080626 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.080737 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.084357 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:24.084363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:24.084366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:24.084402 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:24.087588 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.088422 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.088426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.088429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.088564 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.089183 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.089189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.089193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.089661 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:24.089784 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:24.090569 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 08:57:24.090804 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 08:57:24.092093 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.073165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43399 (local address 127.22.98.3:60379)
0806 08:57:24.074082 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:24.074141 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:24.075841 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:24.076542 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:24.076551 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:24.077085 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:24.077684 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:24.077702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.078746 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.078750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:24.080086 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:24.080100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.080855 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.080863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:24.081093 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.081756 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:24.081783 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:24.084108 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:24.087738 (+ 3630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:24.087746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:24.087762 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:24.088270 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:24.088710 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:24.088715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:24.088718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:24.089022 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:24.089799 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:24.089806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:24.090170 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 08:57:24.090434 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 08:57:24.090867 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":376,"threads_started":1}
I20260806 08:57:24.093564 28109 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43399
I20260806 08:57:24.093850 28109 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:24.094568 28109 heartbeater.cc:511] Master 127.22.98.62:43399 requested a full tablet report, sending...
I20260806 08:57:24.096120 27442 ts_manager.cc:194] Registered new tserver with Master: 6d7b0b69403443458f622534e92cda33 (127.22.98.3:39449)
I20260806 08:57:24.096917 27442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:60379
I20260806 08:57:24.099556 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:24.154225 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.139859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.22.98.62:43399)
0806 08:57:24.140113 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:24.140120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:24.140397 (+ 277us) server_negotiation.cc:407] Connection header received
0806 08:57:24.140547 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:24.140551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:24.140620 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:24.140703 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:24.142167 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.142983 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.144353 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.144857 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.144971 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.148984 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:24.148990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:24.148993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:24.149033 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:24.151417 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.152156 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.152161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.152163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.152314 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.152789 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.152793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.152795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.153154 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:24.153375 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:24.153721 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 08:57:24.153901 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 08:57:24.154851 27442 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33040
I20260806 08:57:24.156430 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27532
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:24.172158 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.157531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.22.98.1:35493)
0806 08:57:24.158140 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:24.158147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:24.158171 (+ 24us) server_negotiation.cc:407] Connection header received
0806 08:57:24.158224 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:24.158231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:24.158429 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:24.158568 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:24.159788 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.160800 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.161988 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.162562 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.162686 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.166745 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:24.166757 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:24.166774 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:24.166811 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:24.169567 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.170230 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.170237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.170239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.170383 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.170767 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.170770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.170772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.171125 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:24.171387 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:24.171642 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 08:57:24.171861 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":305,"threads_started":1}
W20260806 08:57:24.173448 27726 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47988
W20260806 08:57:24.174685 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27532: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:24.174890 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27532
I20260806 08:57:24.198273 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27742
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:24.215008 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.199215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.22.98.2:43575)
0806 08:57:24.199797 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:24.199804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:24.199831 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:57:24.199895 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:24.199902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:24.200118 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:24.200277 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:24.201593 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.202643 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.204089 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.204683 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.204803 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.209029 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:24.209043 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:24.209062 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:24.209121 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:24.212164 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.212838 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.212846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.212848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.212984 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.213355 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.213359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.213361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.213730 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:24.214031 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:24.214387 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 08:57:24.214664 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":251,"threads_started":1}
W20260806 08:57:24.216389 27920 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47604
W20260806 08:57:24.217619 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27742: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:24.217789 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27742
I20260806 08:57:24.240588 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27937
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[27375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:24.258589 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:24.241702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.22.98.3:39449)
0806 08:57:24.242422 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:24.242429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:24.242462 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:57:24.242532 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:24.242539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:24.242770 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:24.242923 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:24.244436 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.245560 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.247021 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:24.247802 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:24.247911 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:24.252498 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:24.252513 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:24.252531 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:24.252571 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:24.255427 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.256366 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.256375 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.256378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.256555 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:24.257005 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:24.257010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:24.257012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:24.257423 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:24.257786 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:24.258051 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 08:57:24.258272 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":399,"threads_started":1}
W20260806 08:57:24.259922 28100 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48494
W20260806 08:57:24.261008 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27937: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 08:57:24.261175 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27937
I20260806 08:57:24.283102 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 27394
I20260806 08:57:24.284433 27491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:24.387265 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 27394
2026-08-06T08:57:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3840 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[28133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[28133](info): set up 2 sockets
Aug 06 08:57:24 dist-test-slave-jpj1 krb5kdc[28133](info): commencing operation
krb5kdc: starting...
W20260806 08:57:25.576246 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:25 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:25.811894 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35713
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:35713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:26.154772 28152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:26.155081 28152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:26.155208 28152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:26.162647 28152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:26.162801 28152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:26.162896 28152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:26.162961 28152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:26.163009 28152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:26.173349 28152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:35713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:26.174739 28152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:26.176191 28152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:26.188122 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:26.188095 28161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:26.188166 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:26.189321 28152 server_base.cc:1034] running on GCE node
I20260806 08:57:26.190245 28152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:26.191850 28152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:26.193328 28152 hybrid_clock.cc:648] HybridClock initialized: now 1786006646193246 us; error 57 us; skew 500 ppm
I20260806 08:57:26.193749 28152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:26 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:26.200971 28152 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:26.202778 28152 webserver.cc:460] Webserver started at http://127.22.98.62:38745/ using document root <none> and password file <none>
I20260806 08:57:26.203612 28152 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:26.203770 28152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:26.204137 28152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:26.207500 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "7371d93c46e349a6a3e46d3d81219d15"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "6c748c907f4ffb9f38cb8b5370bd4c95"
server_key_iv: "1185f65e8e26fd4092db11b849771119"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.208446 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "7371d93c46e349a6a3e46d3d81219d15"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "6c748c907f4ffb9f38cb8b5370bd4c95"
server_key_iv: "1185f65e8e26fd4092db11b849771119"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.214272 28152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:57:26.218398 28173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:26.220021 28152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:26.220227 28152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "7371d93c46e349a6a3e46d3d81219d15"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "6c748c907f4ffb9f38cb8b5370bd4c95"
server_key_iv: "1185f65e8e26fd4092db11b849771119"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.220458 28152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:26.245499 28152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:26.250772 28152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:26.251128 28152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:26.270155 28152 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:35713
I20260806 08:57:26.270215 28240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:35713 every 8 connection(s)
I20260806 08:57:26.271623 28152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:26.275408 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 28152
I20260806 08:57:26.275581 28241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:26.275705 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:26.276330 22920 external_mini_cluster.cc:1428] Setting key 465ea6ba5565d1b512e1a1795a9766bf
I20260806 08:57:26.289532 28241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15: Bootstrap starting.
Aug 06 08:57:26 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:26.293355 28241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:26.294589 28241 log.cc:824] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:26.297640 28241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15: No bootstrap required, opened a new log
I20260806 08:57:26.300748 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:26.278367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.22.98.62:35713)
0806 08:57:26.279174 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:26.279186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:26.279230 (+ 44us) server_negotiation.cc:407] Connection header received
0806 08:57:26.281542 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:26.281571 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:26.282035 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:26.282402 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:26.284402 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.285785 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.287614 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.288373 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.288515 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:26.293108 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:26.293117 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:26.293135 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:26.293179 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:26.296389 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:26.297131 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:26.297139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:26.297146 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:26.297330 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:26.297719 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:26.297724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:26.297726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:26.298330 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:26.298634 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:26.299107 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 08:57:26.299651 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":370,"threads_started":1}
I20260806 08:57:26.306156 28241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } }
I20260806 08:57:26.306578 28241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:26.306691 28241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7371d93c46e349a6a3e46d3d81219d15, State: Initialized, Role: FOLLOWER
I20260806 08:57:26.307379 28241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } }
I20260806 08:57:26.307682 28241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:26.307832 28241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:26.307986 28241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:26.310170 28241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } }
I20260806 08:57:26.310689 28241 leader_election.cc:302] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7371d93c46e349a6a3e46d3d81219d15; no voters:
I20260806 08:57:26.311278 28241 leader_election.cc:288] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:26.311431 28246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:26.312187 28246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [term 1 LEADER]: Becoming Leader. State: Replica: 7371d93c46e349a6a3e46d3d81219d15, State: Running, Role: LEADER
I20260806 08:57:26.312783 28246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } }
I20260806 08:57:26.313452 28241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:26.316123 28248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7371d93c46e349a6a3e46d3d81219d15. Latest consensus state: current_term: 1 leader_uuid: "7371d93c46e349a6a3e46d3d81219d15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } } }
I20260806 08:57:26.316074 28247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7371d93c46e349a6a3e46d3d81219d15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7371d93c46e349a6a3e46d3d81219d15" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35713 } } }
I20260806 08:57:26.316658 28248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:26.316841 28247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:26.317665 28258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:26.323127 28258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:26.332510 28258 catalog_manager.cc:1347] Generated new cluster ID: b264dd52d90246359775b4b1706563c1
I20260806 08:57:26.332649 28258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:26.356001 28258 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:26.357991 28258 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:26.368806 28258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7371d93c46e349a6a3e46d3d81219d15: Generated new TSK 0
I20260806 08:57:26.369709 28258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:26.437505 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:26.781956 28276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:26.782305 28276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:26.782459 28276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:26.790071 28276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:26.790221 28276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:26.790382 28276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:26.800670 28276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:26.802083 28276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:26.803490 28276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:26.817360 28286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:26.817389 28289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:26.817421 28284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:26.818377 28276 server_base.cc:1034] running on GCE node
I20260806 08:57:26.819017 28276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:26.820108 28276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:26.821337 28276 hybrid_clock.cc:648] HybridClock initialized: now 1786006646821281 us; error 49 us; skew 500 ppm
I20260806 08:57:26.821677 28276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:26 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:26.825095 28276 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:26.826685 28276 webserver.cc:460] Webserver started at http://127.22.98.1:45675/ using document root <none> and password file <none>
I20260806 08:57:26.827404 28276 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:26.827574 28276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:26.827903 28276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:26.831017 28276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "5c2d646741d84c7391e4b5e2ec0df2c1"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "a16eff22b4b30bdb5d4680e6761ba656"
server_key_iv: "893030704eb103205349c6acf4f04780"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.831900 28276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "5c2d646741d84c7391e4b5e2ec0df2c1"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "a16eff22b4b30bdb5d4680e6761ba656"
server_key_iv: "893030704eb103205349c6acf4f04780"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.837525 28276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:26.841568 28299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:26.843063 28276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 08:57:26.843245 28276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "5c2d646741d84c7391e4b5e2ec0df2c1"
format_stamp: "Formatted at 2026-08-06 08:57:26 on dist-test-slave-jpj1"
server_key: "a16eff22b4b30bdb5d4680e6761ba656"
server_key_iv: "893030704eb103205349c6acf4f04780"
server_key_version: "encryptionkey@0"
I20260806 08:57:26.843453 28276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:26.859522 28276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:26.864071 28276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:26.864357 28276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:26.865425 28276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:26.866838 28276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:26.866914 28276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:26.867019 28276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:26.867076 28276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:26.899971 28276 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:44371
I20260806 08:57:26.900004 28458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:44371 every 8 connection(s)
I20260806 08:57:26.901347 28276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:26.902770 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 28276
I20260806 08:57:26.902992 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:26.903672 22920 external_mini_cluster.cc:1428] Setting key 8b44d5089e9921f1776caacc5c318c7c
Aug 06 08:57:26 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:26.923314 28464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:26.904239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:43881 (local address 127.22.98.62:35713)
0806 08:57:26.904685 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:26.904691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:26.905890 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 08:57:26.907844 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:26.907849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:26.907930 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:26.908025 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:26.909635 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.910367 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.912040 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.912481 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.912601 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:26.916840 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:26.916848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:26.916853 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:26.916894 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:26.920011 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:26.920808 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:26.920813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:26.920817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:26.920969 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:26.921401 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:26.921406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:26.921409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:26.921811 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:26.921950 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:26.922693 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 08:57:26.922931 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260806 08:57:26.924259 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:26.904839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35713 (local address 127.22.98.1:43881)
0806 08:57:26.905720 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:26.905773 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:26.907555 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:26.908300 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:26.908309 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:26.908852 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:26.909448 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:26.909467 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.910537 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.910543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:26.911883 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:26.911897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:26.912742 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:26.912751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:26.913004 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:26.913791 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:26.913821 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:26.916571 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:26.920168 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:26.920178 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:26.920194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:26.920663 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:26.921066 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:26.921069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:26.921071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:26.921286 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:26.921931 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:26.921937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:26.922284 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 08:57:26.922524 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 08:57:26.923008 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":320,"threads_started":1}
I20260806 08:57:26.925897 28461 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35713
I20260806 08:57:26.926199 28461 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:26.926949 28461 heartbeater.cc:511] Master 127.22.98.62:35713 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
I20260806 08:57:26.929282 28198 ts_manager.cc:194] Registered new tserver with Master: 5c2d646741d84c7391e4b5e2ec0df2c1 (127.22.98.1:44371)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
I20260806 08:57:26.930977 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:43881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:26.957237 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:27.301010 28470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:27.301332 28470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:27.301457 28470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:27.308759 28470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:27.308877 28470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:27.309036 28470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:27.319110 28470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:27.320519 28470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:27.321898 28470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:27.335565 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:27.335606 28485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:27.335655 28477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:27.336637 28470 server_base.cc:1034] running on GCE node
I20260806 08:57:27.337347 28470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:27.338351 28470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:27.339600 28470 hybrid_clock.cc:648] HybridClock initialized: now 1786006647339510 us; error 48 us; skew 500 ppm
I20260806 08:57:27.339995 28470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:27 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:27.343483 28470 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:27.345211 28470 webserver.cc:460] Webserver started at http://127.22.98.2:34325/ using document root <none> and password file <none>
I20260806 08:57:27.345955 28470 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:27.346081 28470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:27.346419 28470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:27.349592 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "43e399ed2cad49e78fbcb44ee224f56a"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "ec8236afa2ff2ac83089794a366056ee"
server_key_iv: "a8196ba6a90003e395b502aae29c63cc"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.350431 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "43e399ed2cad49e78fbcb44ee224f56a"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "ec8236afa2ff2ac83089794a366056ee"
server_key_iv: "a8196ba6a90003e395b502aae29c63cc"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.356211 28470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:27.360195 28497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.361784 28470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:27.361974 28470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "43e399ed2cad49e78fbcb44ee224f56a"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "ec8236afa2ff2ac83089794a366056ee"
server_key_iv: "a8196ba6a90003e395b502aae29c63cc"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.362210 28470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:27.380885 28470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:27.385468 28470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:27.385775 28470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:27.386837 28470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:27.388383 28470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:27.388470 28470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.388594 28470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:27.388648 28470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.425108 28470 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:37049
I20260806 08:57:27.425141 28662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:37049 every 8 connection(s)
I20260806 08:57:27.426740 28470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:27.430935 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 28470
I20260806 08:57:27.431154 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:27.431830 22920 external_mini_cluster.cc:1428] Setting key c6a81c8588d500e21aa353601c4a7cc4
Aug 06 08:57:27 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:27.455053 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:27.429874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:34025 (local address 127.22.98.62:35713)
0806 08:57:27.430337 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:27.430342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:27.431598 (+ 1256us) server_negotiation.cc:407] Connection header received
0806 08:57:27.433990 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:27.433994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:27.434061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:27.434286 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:27.435999 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:27.436720 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:27.438268 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:27.438698 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:27.438809 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:27.445675 (+ 6866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:27.445683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:27.445688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:27.445733 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:27.450014 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:27.451576 (+ 1562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:27.451582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:27.451585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:27.451800 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:27.452550 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:27.452555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:27.452559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:27.452974 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:27.453150 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:27.454375 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0806 08:57:27.454649 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
I20260806 08:57:27.456125 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:27.430457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35713 (local address 127.22.98.2:34025)
0806 08:57:27.431418 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:27.431459 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:27.433662 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:27.434596 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:27.434606 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:27.435156 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:27.435826 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:27.435849 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:27.436875 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:27.436879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:27.438124 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:27.438137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:27.438975 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:27.438984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:27.439190 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:27.442589 (+ 3399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:27.442629 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:27.445276 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:27.450263 (+ 4987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:27.450274 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:27.450307 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:27.451281 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:27.452031 (+ 750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:27.452037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:27.452040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:27.452381 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:27.453148 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:27.453157 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:27.453900 (+ 743us) client_negotiation.cc:769] Sending connection context
0806 08:57:27.454205 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 08:57:27.454827 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":427,"threads_started":1}
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
I20260806 08:57:27.457890 28663 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35713
I20260806 08:57:27.458220 28663 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:27.459033 28663 heartbeater.cc:511] Master 127.22.98.62:35713 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
I20260806 08:57:27.460731 28198 ts_manager.cc:194] Registered new tserver with Master: 43e399ed2cad49e78fbcb44ee224f56a (127.22.98.2:37049)
I20260806 08:57:27.461892 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:34025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:27.492043 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:27.851979 28670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:27.852281 28670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:27.852402 28670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:27.859995 28670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:27.860122 28670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:27.860283 28670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:27.870436 28670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:27.871874 28670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:27.873250 28670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:27.886725 28677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:27.886765 28678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:27.886797 28680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:27.888007 28670 server_base.cc:1034] running on GCE node
I20260806 08:57:27.888754 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:27.889796 28670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:27.891021 28670 hybrid_clock.cc:648] HybridClock initialized: now 1786006647890966 us; error 50 us; skew 500 ppm
I20260806 08:57:27.891396 28670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:27 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:27.894672 28670 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:27.896404 28670 webserver.cc:460] Webserver started at http://127.22.98.3:45099/ using document root <none> and password file <none>
I20260806 08:57:27.897130 28670 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:27.897269 28670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:27.897603 28670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:27.900764 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "549ee33b28284387a277fe66c868a559"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "40ee0a9aec5cbd50274e28a2e3fad26f"
server_key_iv: "2976e132b52ce5d2248b3c9697235233"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.901548 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "549ee33b28284387a277fe66c868a559"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "40ee0a9aec5cbd50274e28a2e3fad26f"
server_key_iv: "2976e132b52ce5d2248b3c9697235233"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.906927 28670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 08:57:27.910557 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.912087 28670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:27.912266 28670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "549ee33b28284387a277fe66c868a559"
format_stamp: "Formatted at 2026-08-06 08:57:27 on dist-test-slave-jpj1"
server_key: "40ee0a9aec5cbd50274e28a2e3fad26f"
server_key_iv: "2976e132b52ce5d2248b3c9697235233"
server_key_version: "encryptionkey@0"
I20260806 08:57:27.912465 28670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:27.934218 28461 heartbeater.cc:503] Master 127.22.98.62:35713 was elected leader, sending a full tablet report...
I20260806 08:57:27.952524 28670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:27.957055 28670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:27.957330 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:27.958317 28670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:27.959786 28670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:27.959863 28670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.959960 28670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:27.960003 28670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:27.990373 28670 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:41751
I20260806 08:57:27.990446 28853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:41751 every 8 connection(s)
I20260806 08:57:27.991773 28670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:27.996799 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 28670
I20260806 08:57:27.997004 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:27.997531 22920 external_mini_cluster.cc:1428] Setting key 6ac420b0c676977a0d640288c9d0f845
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:28.013870 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:27.994401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:56731 (local address 127.22.98.62:35713)
0806 08:57:27.994879 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:27.994884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:27.995884 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 08:57:27.998334 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:27.998340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:27.998526 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:27.998658 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.000560 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.001427 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.003035 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.003436 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.003559 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.006991 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.006998 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.007003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.007043 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.009973 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.010897 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.010903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.010906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.011085 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.011723 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.011729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.011732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.012073 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.012224 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.013186 (+ 962us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.013487 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":212,"threads_started":1}
I20260806 08:57:28.015005 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:27.994967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35713 (local address 127.22.98.3:56731)
0806 08:57:27.995714 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:27.995762 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:27.998002 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:27.998978 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:27.998992 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:27.999761 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:28.000383 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:28.000403 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.001611 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.001616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:28.002902 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:28.002915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.003650 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.003656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:28.003866 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.004490 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:28.004513 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:28.006744 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:28.010166 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:28.010176 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:28.010195 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:28.010732 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:28.011246 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:28.011250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:28.011253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:28.011593 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:28.012258 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:28.012268 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:28.012706 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 08:57:28.013045 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 08:57:28.013710 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":298,"threads_started":1}
I20260806 08:57:28.016809 28854 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35713
I20260806 08:57:28.017184 28854 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:28.018091 28854 heartbeater.cc:511] Master 127.22.98.62:35713 requested a full tablet report, sending...
I20260806 08:57:28.019827 28198 ts_manager.cc:194] Registered new tserver with Master: 549ee33b28284387a277fe66c868a559 (127.22.98.3:41751)
I20260806 08:57:28.020675 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:56731
I20260806 08:57:28.026517 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:28.085816 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:28.069056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.22.98.1:44371)
0806 08:57:28.069788 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:28.069796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:28.069825 (+ 29us) server_negotiation.cc:407] Connection header received
0806 08:57:28.069891 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:28.069898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:28.070136 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:28.070274 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.071833 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.072957 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.074304 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.074907 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.075016 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.079318 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.079335 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.079353 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.079398 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.082495 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.083355 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.083364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.083366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.083522 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.084091 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.084096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.084098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.084560 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.084829 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.085168 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.085429 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":357,"threads_started":1}
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:28.106375 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:28.091306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.22.98.62:35713)
0806 08:57:28.091559 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:28.091567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:28.091789 (+ 222us) server_negotiation.cc:407] Connection header received
0806 08:57:28.091977 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:28.091984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:28.092059 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:28.092146 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.093511 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.094265 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.095585 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.096026 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.096153 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.100522 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.100528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.100531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.100576 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.103452 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.104306 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.104311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.104313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.104461 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.104848 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.104852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.104854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.105213 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.105464 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.105805 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.106044 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 08:57:28.106997 28198 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46486
I20260806 08:57:28.108222 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 28276
I20260806 08:57:28.122459 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:28.109397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60346 (local address 127.22.98.1:44371)
0806 08:57:28.109654 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:28.109661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:28.109751 (+ 90us) server_negotiation.cc:407] Connection header received
0806 08:57:28.109936 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:28.109941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:28.110008 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:28.110093 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.111409 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.112152 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.113468 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.113993 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.114136 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.116843 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.116848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.116851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.116890 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.119743 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.120465 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.120471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.120474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.120618 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.121009 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.121015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.121017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.121307 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.121546 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.121896 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.122146 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 08:57:28.123275 28453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:28.206512 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 28276
I20260806 08:57:28.232896 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 28470
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:28.250293 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:28.233861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.22.98.2:37049)
0806 08:57:28.234455 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:28.234463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:28.234492 (+ 29us) server_negotiation.cc:407] Connection header received
0806 08:57:28.234565 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:28.234572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:28.234794 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:28.234937 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.236219 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.237297 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.238678 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.239283 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.239404 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.244147 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.244164 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.244183 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.244223 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.247162 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.247974 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.247982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.247984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.248124 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.248533 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.248537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.248539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.248939 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.249250 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.249636 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.249925 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":306,"threads_started":1}
I20260806 08:57:28.251726 28653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:28.335111 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 28470
I20260806 08:57:28.358220 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 28670
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:28.376765 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:28.359401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.22.98.3:41751)
0806 08:57:28.360173 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:28.360181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:28.360213 (+ 32us) server_negotiation.cc:407] Connection header received
0806 08:57:28.360275 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:28.360288 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:28.360503 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:28.360646 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:28.362014 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.363153 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.364726 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:28.365362 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:28.365474 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:28.370182 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:28.370198 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:28.370223 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:28.370275 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:28.373333 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.374141 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.374150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.374153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.374312 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:28.374871 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:28.374876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:28.374879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:28.375303 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:28.375625 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:28.376107 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 08:57:28.376350 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":434,"threads_started":1}
I20260806 08:57:28.378269 28849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:28.461727 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 28670
I20260806 08:57:28.484099 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 28152
I20260806 08:57:28.485419 28234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:28.591574 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 28152
2026-08-06T08:57:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4199 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28880](info): set up 2 sockets
Aug 06 08:57:28 dist-test-slave-jpj1 krb5kdc[28880](info): commencing operation
krb5kdc: starting...
W20260806 08:57:29.798095 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:29 dist-test-slave-jpj1 krb5kdc[28880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:30.034817 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34543
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:30.373226 28893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:30.373572 28893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:30.373728 28893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:30.381089 28893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:30.381238 28893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:30.381321 28893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:30.381400 28893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:30.381469 28893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:30.391638 28893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:30.392964 28893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:30.394307 28893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:30.405869 28899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:30.405927 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:30.405948 28900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:30.406822 28893 server_base.cc:1034] running on GCE node
I20260806 08:57:30.407688 28893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:30.409140 28893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:30.410341 28893 hybrid_clock.cc:648] HybridClock initialized: now 1786006650410290 us; error 38 us; skew 500 ppm
I20260806 08:57:30.410696 28893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:30 dist-test-slave-jpj1 krb5kdc[28880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:30.413790 28893 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:30.414921 28893 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:30.415026 28893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:30.415372 28893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:30.418421 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "49199a15c9cf797eacdbcdeb892975b1"
server_key_iv: "02e307f40aada7f3dc75a9c37a1def24"
server_key_version: "encryptionkey@0"
I20260806 08:57:30.419194 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "49199a15c9cf797eacdbcdeb892975b1"
server_key_iv: "02e307f40aada7f3dc75a9c37a1def24"
server_key_version: "encryptionkey@0"
I20260806 08:57:30.424278 28893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:30.428038 28914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:30.429520 28893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 08:57:30.429721 28893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "49199a15c9cf797eacdbcdeb892975b1"
server_key_iv: "02e307f40aada7f3dc75a9c37a1def24"
server_key_version: "encryptionkey@0"
I20260806 08:57:30.429942 28893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:30.444195 28893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:30.449287 28893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:30.449617 28893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:30.467494 28893 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34543
I20260806 08:57:30.467514 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34543 every 8 connection(s)
I20260806 08:57:30.468396 28893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:30.468999 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 28893
I20260806 08:57:30.469249 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:30.469937 22920 external_mini_cluster.cc:1428] Setting key 6333b03fe3e5535486f1e7c1a3035f9b
I20260806 08:57:30.472885 28988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 08:57:30 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:30.486327 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: Bootstrap starting.
I20260806 08:57:30.489847 28988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:30.490945 28988 log.cc:824] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:30.493022 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:30.471922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.22.98.62:34543)
0806 08:57:30.472702 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:30.472715 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:30.472756 (+ 41us) server_negotiation.cc:407] Connection header received
0806 08:57:30.474383 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:30.474405 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:30.474834 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:30.475150 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:30.477004 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:30.478402 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:30.480223 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:30.480956 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:30.481089 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:30.485892 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:30.485901 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:30.485920 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:30.485957 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:30.488874 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:30.489674 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:30.489681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:30.489688 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:30.489886 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:30.490265 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:30.490268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:30.490270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:30.490818 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:30.491109 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:30.491601 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 08:57:30.492035 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":363,"threads_started":1}
I20260806 08:57:30.493899 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: No bootstrap required, opened a new log
I20260806 08:57:30.501832 28988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } }
I20260806 08:57:30.502231 28988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:30.502329 28988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f08e541a2f455fb6ed0bdaf9e7bb20, State: Initialized, Role: FOLLOWER
I20260806 08:57:30.502882 28988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } }
I20260806 08:57:30.503113 28988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:30.503249 28988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:30.503412 28988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:30.505684 28988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } }
I20260806 08:57:30.506225 28988 leader_election.cc:302] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76f08e541a2f455fb6ed0bdaf9e7bb20; no voters:
I20260806 08:57:30.506839 28988 leader_election.cc:288] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:30.506943 28994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:30.508188 28994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [term 1 LEADER]: Becoming Leader. State: Replica: 76f08e541a2f455fb6ed0bdaf9e7bb20, State: Running, Role: LEADER
I20260806 08:57:30.508734 28994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } }
I20260806 08:57:30.509192 28988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:30.511575 28997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76f08e541a2f455fb6ed0bdaf9e7bb20. Latest consensus state: current_term: 1 leader_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } } }
I20260806 08:57:30.511997 28997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:30.512544 28995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f08e541a2f455fb6ed0bdaf9e7bb20" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34543 } } }
I20260806 08:57:30.512871 28995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:30.512980 29006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:30.520226 29006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:30.529618 29006 catalog_manager.cc:1347] Generated new cluster ID: 6728f2ad917f44de8ad81e269c1ad8ec
I20260806 08:57:30.529754 29006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:30.547785 29006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:30.549149 29006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:30.555373 29006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: Generated new TSK 0
I20260806 08:57:30.556259 29006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:30.616495 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:30.950716 29021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:30.951005 29021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:30.951102 29021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:30.958259 29021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:30.958377 29021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:30.958501 29021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:30.968132 29021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:30.969391 29021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:30.970737 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:30.983395 29029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:30.983353 29030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:30.983423 29035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:30.984325 29021 server_base.cc:1034] running on GCE node
I20260806 08:57:30.984933 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:30.985893 29021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:30.987111 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786006650987048 us; error 55 us; skew 500 ppm
I20260806 08:57:30.987466 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:30 dist-test-slave-jpj1 krb5kdc[28880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:30.990619 29021 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:30.991979 29021 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:30.992085 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:30.992471 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:30.995453 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "f582980cffaf4648be4f5c0708bcc096"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "c5e38947cef01e5d08a909ad45ca9e9e"
server_key_iv: "b3d4758dad1707276485ff8319c17955"
server_key_version: "encryptionkey@0"
I20260806 08:57:30.996325 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "f582980cffaf4648be4f5c0708bcc096"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "c5e38947cef01e5d08a909ad45ca9e9e"
server_key_iv: "b3d4758dad1707276485ff8319c17955"
server_key_version: "encryptionkey@0"
I20260806 08:57:31.001806 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 08:57:31.005582 29046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.007071 29021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:31.007241 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "f582980cffaf4648be4f5c0708bcc096"
format_stamp: "Formatted at 2026-08-06 08:57:30 on dist-test-slave-jpj1"
server_key: "c5e38947cef01e5d08a909ad45ca9e9e"
server_key_iv: "b3d4758dad1707276485ff8319c17955"
server_key_version: "encryptionkey@0"
I20260806 08:57:31.007457 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:31.029112 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:31.033244 29021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:31.033502 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:31.034523 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:31.035975 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:31.036065 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.036151 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:31.036208 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.068198 29021 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:34199
I20260806 08:57:31.068274 29211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:34199 every 8 connection(s)
I20260806 08:57:31.069169 29021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:31.079322 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29021
I20260806 08:57:31.079564 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:31.080204 22920 external_mini_cluster.cc:1428] Setting key efc9a36de4da3477228323876fe0b4b4
Aug 06 08:57:31 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:31.089370 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:31.071820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:51069 (local address 127.22.98.62:34543)
0806 08:57:31.072269 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:31.072274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:31.073193 (+ 919us) server_negotiation.cc:407] Connection header received
0806 08:57:31.075037 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:31.075043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:31.075125 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:31.075222 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:31.076903 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.077630 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.079232 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.079761 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.079880 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:31.083348 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:31.083354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:31.083357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:31.083390 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:31.086099 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:31.086896 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:31.086902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:31.086905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:31.087036 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:31.087487 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:31.087491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:31.087494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:31.087831 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:31.087971 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:31.088742 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 08:57:31.088995 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":227,"threads_started":1}
I20260806 08:57:31.090442 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:31.072298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34543 (local address 127.22.98.1:51069)
0806 08:57:31.073027 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:31.073080 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:31.074746 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:31.075494 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:31.075502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:31.076112 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:31.076717 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:31.076736 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.077800 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.077804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:31.079076 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:31.079090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.080001 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.080008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:31.080232 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:31.080891 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:31.080913 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:31.083114 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:31.086264 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:31.086275 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:31.086294 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:31.086771 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:31.087158 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:31.087163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:31.087175 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:31.087365 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:31.087971 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:31.087980 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:31.088354 (+ 374us) client_negotiation.cc:769] Sending connection context
0806 08:57:31.088614 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 08:57:31.089215 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":285,"threads_started":1}
I20260806 08:57:31.092448 29217 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34543
I20260806 08:57:31.092792 29217 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:31.093500 29217 heartbeater.cc:511] Master 127.22.98.62:34543 requested a full tablet report, sending...
I20260806 08:57:31.096088 28936 ts_manager.cc:194] Registered new tserver with Master: f582980cffaf4648be4f5c0708bcc096 (127.22.98.1:34199)
I20260806 08:57:31.098141 28936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:51069
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:31.132894 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:31.473825 29227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:31.474212 29227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:31.474368 29227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:31.485123 29227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:31.485235 29227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:31.485378 29227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:31.495208 29227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:31.496466 29227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:31.497735 29227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:31.510380 29234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:31.510408 29238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:31.510367 29235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:31.511191 29227 server_base.cc:1034] running on GCE node
I20260806 08:57:31.511854 29227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:31.512810 29227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:31.514060 29227 hybrid_clock.cc:648] HybridClock initialized: now 1786006651514007 us; error 38 us; skew 500 ppm
I20260806 08:57:31.514433 29227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:31 dist-test-slave-jpj1 krb5kdc[28880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006651, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:31.520901 29227 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:31.522225 29227 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:31.522322 29227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:31.522610 29227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:31.525650 29227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "646210a54ecd428f8486486ce1688c86"
format_stamp: "Formatted at 2026-08-06 08:57:31 on dist-test-slave-jpj1"
server_key: "d049cc4033400b0a2267a6b1c61bfc6d"
server_key_iv: "9f20d29d85858d95b010ee1caca13ab7"
server_key_version: "encryptionkey@0"
I20260806 08:57:31.526441 29227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "646210a54ecd428f8486486ce1688c86"
format_stamp: "Formatted at 2026-08-06 08:57:31 on dist-test-slave-jpj1"
server_key: "d049cc4033400b0a2267a6b1c61bfc6d"
server_key_iv: "9f20d29d85858d95b010ee1caca13ab7"
server_key_version: "encryptionkey@0"
I20260806 08:57:31.531565 29227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 08:57:31.535229 29245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.536592 29227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:31.536748 29227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "646210a54ecd428f8486486ce1688c86"
format_stamp: "Formatted at 2026-08-06 08:57:31 on dist-test-slave-jpj1"
server_key: "d049cc4033400b0a2267a6b1c61bfc6d"
server_key_iv: "9f20d29d85858d95b010ee1caca13ab7"
server_key_version: "encryptionkey@0"
I20260806 08:57:31.536945 29227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:31.553386 29227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:31.557595 29227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:31.557849 29227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:31.558797 29227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:31.560276 29227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:31.560372 29227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.560479 29227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:31.560554 29227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:31.619299 29227 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:38389
I20260806 08:57:31.619406 29409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:38389 every 8 connection(s)
I20260806 08:57:31.620250 29227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:31.626868 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29227
I20260806 08:57:31.627065 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:31.627496 22920 external_mini_cluster.cc:1428] Setting key fa63e66a196a2120084d8c9bec31d647
Aug 06 08:57:31 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006651, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:31.644935 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:31.623182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:58755 (local address 127.22.98.62:34543)
0806 08:57:31.623707 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:31.623714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:31.624885 (+ 1171us) server_negotiation.cc:407] Connection header received
0806 08:57:31.627649 (+ 2764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:31.627655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:31.627794 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:31.627950 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:31.630128 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.631184 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.633661 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.634304 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.634482 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:31.638868 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:31.638873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:31.638876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:31.638908 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:31.641847 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:31.642583 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:31.642588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:31.642590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:31.642737 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:31.643150 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:31.643154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:31.643156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:31.643428 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:31.643584 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:31.644312 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 08:57:31.644531 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":241,"threads_started":1}
I20260806 08:57:31.646056 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:31.623791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34543 (local address 127.22.98.2:58755)
0806 08:57:31.624668 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:31.624731 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:31.627246 (+ 2515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:31.628221 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:31.628234 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:31.629041 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:31.629908 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:31.629941 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.631350 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.631357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:31.633495 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:31.633513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:31.634565 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:31.634573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:31.634920 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:31.635726 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:31.635760 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:31.638591 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:31.641981 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:31.641991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:31.642007 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:31.642456 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:31.642852 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:31.642857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:31.642859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:31.643035 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:31.643582 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:31.643590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:31.643929 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 08:57:31.644178 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 08:57:31.644756 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":338,"threads_started":1}
I20260806 08:57:31.647578 29410 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34543
I20260806 08:57:31.647898 29410 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:31.648618 29410 heartbeater.cc:511] Master 127.22.98.62:34543 requested a full tablet report, sending...
I20260806 08:57:31.650192 28936 ts_manager.cc:194] Registered new tserver with Master: 646210a54ecd428f8486486ce1688c86 (127.22.98.2:38389)
I20260806 08:57:31.651005 28936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:58755
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:31.691596 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:32.051931 29418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:32.052278 29418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:32.052383 29418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:32.060137 29418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:32.060281 29418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:32.060413 29418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:32.072520 29418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:32.074316 29418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:32.075949 29418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:32.090014 29425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:32.089983 29428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:32.090056 29426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:32.090939 29418 server_base.cc:1034] running on GCE node
I20260806 08:57:32.091741 29418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:32.092842 29418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:32.094091 29418 hybrid_clock.cc:648] HybridClock initialized: now 1786006652094025 us; error 50 us; skew 500 ppm
I20260806 08:57:32.094471 29418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:32 dist-test-slave-jpj1 krb5kdc[28880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006652, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:32.098225 29418 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:32.099607 29418 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:32.099746 29418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:32.100106 29418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:32.101577 29217 heartbeater.cc:503] Master 127.22.98.62:34543 was elected leader, sending a full tablet report...
I20260806 08:57:32.103664 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "fee70f3159b4400eaa0f240ac20c50e6"
format_stamp: "Formatted at 2026-08-06 08:57:32 on dist-test-slave-jpj1"
server_key: "9a0f4b1e8ea97298cc961f58f33c00b7"
server_key_iv: "cb2f15455688ad4102e00c8c7a18e335"
server_key_version: "encryptionkey@0"
I20260806 08:57:32.104605 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "fee70f3159b4400eaa0f240ac20c50e6"
format_stamp: "Formatted at 2026-08-06 08:57:32 on dist-test-slave-jpj1"
server_key: "9a0f4b1e8ea97298cc961f58f33c00b7"
server_key_iv: "cb2f15455688ad4102e00c8c7a18e335"
server_key_version: "encryptionkey@0"
I20260806 08:57:32.110644 29418 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 08:57:32.115039 29434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:32.116750 29418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:32.116954 29418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "fee70f3159b4400eaa0f240ac20c50e6"
format_stamp: "Formatted at 2026-08-06 08:57:32 on dist-test-slave-jpj1"
server_key: "9a0f4b1e8ea97298cc961f58f33c00b7"
server_key_iv: "cb2f15455688ad4102e00c8c7a18e335"
server_key_version: "encryptionkey@0"
I20260806 08:57:32.117223 29418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:32.139607 29418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:32.144505 29418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:32.144850 29418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:32.145991 29418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:32.147655 29418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:32.147774 29418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:32.147886 29418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:32.147964 29418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:32.180171 29418 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:35341
I20260806 08:57:32.180274 29592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:35341 every 8 connection(s)
I20260806 08:57:32.181173 29418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:32.191373 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29418
I20260806 08:57:32.191644 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:32.192202 22920 external_mini_cluster.cc:1428] Setting key b0256134a48358b2e6bc3572d9162a9d
Aug 06 08:57:32 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006652, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:32.204285 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.183959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:58023 (local address 127.22.98.62:34543)
0806 08:57:32.184517 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.184522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.185556 (+ 1034us) server_negotiation.cc:407] Connection header received
0806 08:57:32.187757 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.187762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.187864 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.187967 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:32.189753 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.190653 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.193200 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.193671 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.193797 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.197765 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:32.197771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:32.197775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:32.197818 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:32.200672 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.201589 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.201594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.201596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.201812 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.202382 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.202386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.202388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.202737 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:32.202882 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.203715 (+ 833us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.203942 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":303,"threads_started":1}
I20260806 08:57:32.205509 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.184593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34543 (local address 127.22.98.3:58023)
0806 08:57:32.185374 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.185424 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.187355 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.188211 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.188221 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.188874 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:32.189533 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.189553 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.190902 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.190908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.192990 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.193007 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.193919 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.193927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.194166 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.194884 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:32.194916 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:32.197457 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:32.200852 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:32.200861 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:32.200879 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:32.201408 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:32.201974 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:32.201979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:32.201981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:32.202215 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:32.202892 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:32.202900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:32.203286 (+ 386us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.203593 (+ 307us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.204111 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":302,"threads_started":1}
I20260806 08:57:32.207578 29595 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34543
I20260806 08:57:32.207988 29595 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:32.208863 29595 heartbeater.cc:511] Master 127.22.98.62:34543 requested a full tablet report, sending...
I20260806 08:57:32.210559 28935 ts_manager.cc:194] Registered new tserver with Master: fee70f3159b4400eaa0f240ac20c50e6 (127.22.98.3:35341)
I20260806 08:57:32.211635 28935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:58023
I20260806 08:57:32.221752 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 08:57:32.241006 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.226769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.22.98.62:34543)
0806 08:57:32.226991 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.226996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.227508 (+ 512us) server_negotiation.cc:407] Connection header received
0806 08:57:32.227693 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.227697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.227774 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.227870 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:32.229448 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.230239 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.231631 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.232130 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.232249 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.234947 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:32.234954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:32.234957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:32.235000 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:32.237809 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.238663 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.238667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.238670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.238837 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.239313 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.239317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.239320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.239733 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:32.239957 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.240457 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.240679 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 08:57:32.245815 28935 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:32.253306 28935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:32.281196 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.267477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.22.98.1:34199)
0806 08:57:32.273228 (+ 5751us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.273238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.273268 (+ 30us) server_negotiation.cc:407] Connection header received
0806 08:57:32.273374 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.273382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.273648 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.273855 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:32.274614 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276203 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.278717 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.280021 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.280147 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.280409 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.280698 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.280883 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5561,"thread_start_us":5398,"threads_started":1}
I20260806 08:57:32.281235 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.267196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:34199 (local address 127.0.0.1:44900)
0806 08:57:32.268374 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.268434 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.268648 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.273879 (+ 5231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.273883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.273909 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:32.274396 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.274408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276342 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.276348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.278519 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.278535 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.280219 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.280225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.280446 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.280505 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.280624 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.280833 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":41,"thread_start_us":6011,"threads_started":1}
I20260806 08:57:32.282778 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.269612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35341 (local address 127.0.0.1:39936)
0806 08:57:32.273522 (+ 3910us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.273539 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.273807 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.274708 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.274712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.274738 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:32.275148 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.275161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276870 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.276875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.279257 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.279268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.281768 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.281775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.281986 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.282046 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.282145 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.282332 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3777,"mutex_wait_us":125,"spinlock_wait_cycles":72576,"thread_start_us":2106,"threads_started":1}
I20260806 08:57:32.282799 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.269940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.22.98.3:35341)
0806 08:57:32.270662 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.270672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.273647 (+ 2975us) server_negotiation.cc:407] Connection header received
0806 08:57:32.274022 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.274030 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.274204 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.274365 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:32.275404 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276691 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.279431 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.280927 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.281060 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.281293 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.282236 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.282457 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":398,"threads_started":1}
I20260806 08:57:32.283691 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.268229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.22.98.2:38389)
0806 08:57:32.272787 (+ 4558us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.272797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.272831 (+ 34us) server_negotiation.cc:407] Connection header received
0806 08:57:32.272955 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.272965 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.273269 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.273491 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:32.274440 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276190 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.279422 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.280936 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.281060 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.281257 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.283168 (+ 1911us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.283356 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4384,"thread_start_us":4217,"threads_started":1}
I20260806 08:57:32.285336 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.267521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:38389 (local address 127.0.0.1:58948)
0806 08:57:32.272034 (+ 4513us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.272056 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.272195 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.273510 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.273518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.273547 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:32.274178 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.274192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.276394 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.276403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.279195 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.279213 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.281399 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.281404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.282002 (+ 598us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.282055 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.282144 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.282332 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4374,"mutex_wait_us":281,"spinlock_wait_cycles":1408,"thread_start_us":890,"threads_started":1}
I20260806 08:57:32.285537 29121 tablet_service.cc:1467] Processing CreateTablet for tablet 081b907659db4ad4bb60d20603172dcc (DEFAULT_TABLE table=test-table [id=f414d32b1d08421b8a7b62ebed952400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:32.287047 29121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081b907659db4ad4bb60d20603172dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:32.287518 29501 tablet_service.cc:1467] Processing CreateTablet for tablet 081b907659db4ad4bb60d20603172dcc (DEFAULT_TABLE table=test-table [id=f414d32b1d08421b8a7b62ebed952400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:32.288936 29501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081b907659db4ad4bb60d20603172dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:32.289077 29317 tablet_service.cc:1467] Processing CreateTablet for tablet 081b907659db4ad4bb60d20603172dcc (DEFAULT_TABLE table=test-table [id=f414d32b1d08421b8a7b62ebed952400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:32.290521 29317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 081b907659db4ad4bb60d20603172dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:32.300084 29620 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Bootstrap starting.
I20260806 08:57:32.302273 29621 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Bootstrap starting.
I20260806 08:57:32.305045 29620 tablet_bootstrap.cc:654] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:32.305877 29622 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Bootstrap starting.
I20260806 08:57:32.306550 29620 log.cc:824] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:32.306947 29621 tablet_bootstrap.cc:654] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:32.308106 29621 log.cc:824] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:32.309682 29622 tablet_bootstrap.cc:654] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:32.310168 29620 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: No bootstrap required, opened a new log
I20260806 08:57:32.310509 29620 ts_tablet_manager.cc:1397] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 08:57:32.310794 29622 log.cc:824] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:32.311405 29621 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: No bootstrap required, opened a new log
I20260806 08:57:32.312301 29621 ts_tablet_manager.cc:1397] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 08:57:32.320076 29622 tablet_bootstrap.cc:492] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: No bootstrap required, opened a new log
I20260806 08:57:32.320397 29622 ts_tablet_manager.cc:1397] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.012s
I20260806 08:57:32.321668 29620 raft_consensus.cc:348] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.322165 29620 raft_consensus.cc:374] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:32.322258 29620 raft_consensus.cc:729] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fee70f3159b4400eaa0f240ac20c50e6, State: Initialized, Role: FOLLOWER
I20260806 08:57:32.323127 29620 consensus_queue.cc:260] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.323591 29621 raft_consensus.cc:348] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.324142 29621 raft_consensus.cc:374] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:32.324247 29621 raft_consensus.cc:729] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f582980cffaf4648be4f5c0708bcc096, State: Initialized, Role: FOLLOWER
I20260806 08:57:32.325356 29621 consensus_queue.cc:260] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.325973 29630 raft_consensus.cc:482] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:32.326254 29630 raft_consensus.cc:504] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.330971 29622 raft_consensus.cc:348] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.331475 29622 raft_consensus.cc:374] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:32.331604 29622 raft_consensus.cc:729] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646210a54ecd428f8486486ce1688c86, State: Initialized, Role: FOLLOWER
I20260806 08:57:32.328548 29630 leader_election.cc:288] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 646210a54ecd428f8486486ce1688c86 (127.22.98.2:38389), fee70f3159b4400eaa0f240ac20c50e6 (127.22.98.3:35341)
I20260806 08:57:32.332374 29622 consensus_queue.cc:260] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.333858 29621 ts_tablet_manager.cc:1428] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Time spent starting tablet: real 0.021s user 0.010s sys 0.005s
I20260806 08:57:32.337395 29595 heartbeater.cc:503] Master 127.22.98.62:34543 was elected leader, sending a full tablet report...
I20260806 08:57:32.339823 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.329472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:59447 (local address 127.22.98.2:38389)
0806 08:57:32.329684 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.329690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.329711 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:57:32.329781 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.329786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.329858 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.329960 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:32.330657 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.331775 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.335167 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.336574 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.336703 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.336815 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.339269 (+ 2454us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.339455 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 08:57:32.340821 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.328845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:38389 (local address 127.22.98.1:59447)
0806 08:57:32.329473 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.329497 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.329645 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.329990 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.329995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.330021 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:32.330460 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.330476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.332050 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.332056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.334948 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.334968 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.338513 (+ 3545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.338521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.338905 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.339016 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.340231 (+ 1215us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.340429 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":1225,"threads_started":1}
I20260806 08:57:32.341686 29620 ts_tablet_manager.cc:1428] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Time spent starting tablet: real 0.031s user 0.014s sys 0.008s
I20260806 08:57:32.347335 29410 heartbeater.cc:503] Master 127.22.98.62:34543 was elected leader, sending a full tablet report...
I20260806 08:57:32.347389 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081b907659db4ad4bb60d20603172dcc" candidate_uuid: "f582980cffaf4648be4f5c0708bcc096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646210a54ecd428f8486486ce1688c86" is_pre_election: true
I20260806 08:57:32.348088 29348 raft_consensus.cc:2454] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f582980cffaf4648be4f5c0708bcc096 in term 0.
I20260806 08:57:32.348290 29622 ts_tablet_manager.cc:1428] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Time spent starting tablet: real 0.028s user 0.015s sys 0.012s
I20260806 08:57:32.348733 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.330711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54415 (local address 127.22.98.3:35341)
0806 08:57:32.330904 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.330910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.341236 (+ 10326us) server_negotiation.cc:407] Connection header received
0806 08:57:32.341614 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.341621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.341716 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.341782 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:32.342424 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.343653 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.346235 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.347480 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.347658 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.347771 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.348339 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.348488 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 08:57:32.349166 29047 leader_election.cc:302] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 646210a54ecd428f8486486ce1688c86, f582980cffaf4648be4f5c0708bcc096; no voters:
I20260806 08:57:32.349783 29630 raft_consensus.cc:2781] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:32.349938 29630 raft_consensus.cc:482] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:32.350060 29630 raft_consensus.cc:3037] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:32.350615 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.330582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35341 (local address 127.22.98.1:54415)
0806 08:57:32.341096 (+ 10514us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:32.341118 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:32.341423 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:32.342004 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:32.342009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:32.342039 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:32.342254 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.342265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.343865 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.343871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.345961 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:32.345980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.347885 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.347893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:32.348104 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.348143 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 08:57:32.348962 (+ 819us) client_negotiation.cc:240] Negotiation successful
0806 08:57:32.349130 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10386,"thread_start_us":15557,"threads_started":1}
I20260806 08:57:32.351661 29532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081b907659db4ad4bb60d20603172dcc" candidate_uuid: "f582980cffaf4648be4f5c0708bcc096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee70f3159b4400eaa0f240ac20c50e6" is_pre_election: true
I20260806 08:57:32.352316 29532 raft_consensus.cc:2454] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f582980cffaf4648be4f5c0708bcc096 in term 0.
I20260806 08:57:32.353367 29630 raft_consensus.cc:504] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.354043 29630 leader_election.cc:288] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [CANDIDATE]: Term 1 election: Requested vote from peers 646210a54ecd428f8486486ce1688c86 (127.22.98.2:38389), fee70f3159b4400eaa0f240ac20c50e6 (127.22.98.3:35341)
I20260806 08:57:32.354473 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081b907659db4ad4bb60d20603172dcc" candidate_uuid: "f582980cffaf4648be4f5c0708bcc096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646210a54ecd428f8486486ce1688c86"
I20260806 08:57:32.354535 29532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "081b907659db4ad4bb60d20603172dcc" candidate_uuid: "f582980cffaf4648be4f5c0708bcc096" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee70f3159b4400eaa0f240ac20c50e6"
I20260806 08:57:32.354677 29348 raft_consensus.cc:3037] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:32.354733 29532 raft_consensus.cc:3037] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:32.357071 29532 raft_consensus.cc:2454] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f582980cffaf4648be4f5c0708bcc096 in term 1.
I20260806 08:57:32.357072 29348 raft_consensus.cc:2454] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f582980cffaf4648be4f5c0708bcc096 in term 1.
I20260806 08:57:32.357481 29047 leader_election.cc:302] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 646210a54ecd428f8486486ce1688c86, f582980cffaf4648be4f5c0708bcc096; no voters:
I20260806 08:57:32.357802 29630 raft_consensus.cc:2781] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:32.357960 29630 raft_consensus.cc:686] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 1 LEADER]: Becoming Leader. State: Replica: f582980cffaf4648be4f5c0708bcc096, State: Running, Role: LEADER
I20260806 08:57:32.358493 29630 consensus_queue.cc:237] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } }
I20260806 08:57:32.363029 28935 catalog_manager.cc:5495] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 reported cstate change: term changed from 0 to 1, leader changed from <none> to f582980cffaf4648be4f5c0708bcc096 (127.22.98.1). New cstate: current_term: 1 leader_uuid: "f582980cffaf4648be4f5c0708bcc096" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f582980cffaf4648be4f5c0708bcc096" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 34199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 } health_report { overall_health: UNKNOWN } } }
W20260806 08:57:32.372357 29411 tablet.cc:2369] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:32.385702 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.380352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.22.98.1:34199)
0806 08:57:32.380574 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.380580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.380600 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:57:32.380758 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.380762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.380859 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.381003 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:32.381404 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.382137 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.383494 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.383941 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.384054 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.384583 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:32.385107 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:32.385258 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.385351 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.385439 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 08:57:32.394382 29532 raft_consensus.cc:1261] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 1 FOLLOWER]: Refusing update from remote peer f582980cffaf4648be4f5c0708bcc096: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:32.394443 29348 raft_consensus.cc:1261] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 1 FOLLOWER]: Refusing update from remote peer f582980cffaf4648be4f5c0708bcc096: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:32.395334 29631 consensus_queue.cc:1035] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "646210a54ecd428f8486486ce1688c86" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 38389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:32.395735 29630 consensus_queue.cc:1035] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fee70f3159b4400eaa0f240ac20c50e6" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:32.405763 29639 mvcc.cc:205] Tried to move back new op lower bound from 7315483248197107712 to 7315483248064241664. Current Snapshot: MvccSnapshot[applied={T|T < 7315483248197107712}]
I20260806 08:57:32.407524 29640 mvcc.cc:205] Tried to move back new op lower bound from 7315483248197107712 to 7315483248064241664. Current Snapshot: MvccSnapshot[applied={T|T < 7315483248197107712}]
I20260806 08:57:32.410362 29641 mvcc.cc:205] Tried to move back new op lower bound from 7315483248197107712 to 7315483248064241664. Current Snapshot: MvccSnapshot[applied={T|T < 7315483248197107712}]
I20260806 08:57:32.425675 28935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:32.426023 28935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:32.430558 28935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76f08e541a2f455fb6ed0bdaf9e7bb20: Sending DeleteTablet for 3 replicas of tablet 081b907659db4ad4bb60d20603172dcc
I20260806 08:57:32.432358 29317 tablet_service.cc:1514] Processing DeleteTablet for tablet 081b907659db4ad4bb60d20603172dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:32 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:58948
I20260806 08:57:32.432415 29121 tablet_service.cc:1514] Processing DeleteTablet for tablet 081b907659db4ad4bb60d20603172dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:32 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:44900
I20260806 08:57:32.432686 29501 tablet_service.cc:1514] Processing DeleteTablet for tablet 081b907659db4ad4bb60d20603172dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:32 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:39936
I20260806 08:57:32.434731 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29021
I20260806 08:57:32.445135 29651 tablet_replica.cc:330] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: stopping tablet replica
I20260806 08:57:32.445958 29651 raft_consensus.cc:2229] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:32.446827 29651 raft_consensus.cc:2258] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:32.447875 29653 tablet_replica.cc:330] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: stopping tablet replica
I20260806 08:57:32.448465 29653 raft_consensus.cc:2229] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:32.449117 29653 raft_consensus.cc:2258] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:32.449976 29651 ts_tablet_manager.cc:1905] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:32.451836 29653 ts_tablet_manager.cc:1905] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:32.455634 29652 tablet_replica.cc:330] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: stopping tablet replica
I20260806 08:57:32.456243 29652 raft_consensus.cc:2229] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:32.457108 29652 raft_consensus.cc:2258] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:32.457482 29651 ts_tablet_manager.cc:1918] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:32.457672 29651 log.cc:1196] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/081b907659db4ad4bb60d20603172dcc
I20260806 08:57:32.458257 29651 ts_tablet_manager.cc:1939] T 081b907659db4ad4bb60d20603172dcc P 646210a54ecd428f8486486ce1688c86: Deleting consensus metadata
I20260806 08:57:32.459642 29653 ts_tablet_manager.cc:1918] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:32.459932 29653 log.cc:1196] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/081b907659db4ad4bb60d20603172dcc
I20260806 08:57:32.460605 29653 ts_tablet_manager.cc:1939] T 081b907659db4ad4bb60d20603172dcc P fee70f3159b4400eaa0f240ac20c50e6: Deleting consensus metadata
I20260806 08:57:32.460527 28915 catalog_manager.cc:4841] TS 646210a54ecd428f8486486ce1688c86 (127.22.98.2:38389): tablet 081b907659db4ad4bb60d20603172dcc (table test-table [id=f414d32b1d08421b8a7b62ebed952400]) successfully deleted
I20260806 08:57:32.462666 28915 catalog_manager.cc:4841] TS fee70f3159b4400eaa0f240ac20c50e6 (127.22.98.3:35341): tablet 081b907659db4ad4bb60d20603172dcc (table test-table [id=f414d32b1d08421b8a7b62ebed952400]) successfully deleted
Aug 06 08:57:32 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:32.463300 29652 ts_tablet_manager.cc:1905] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:32.471051 29652 ts_tablet_manager.cc:1918] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:32.471256 29652 log.cc:1196] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/081b907659db4ad4bb60d20603172dcc
I20260806 08:57:32.471871 29652 ts_tablet_manager.cc:1939] T 081b907659db4ad4bb60d20603172dcc P f582980cffaf4648be4f5c0708bcc096: Deleting consensus metadata
I20260806 08:57:32.473841 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.436824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.22.98.1:34199)
0806 08:57:32.437041 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.437049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.447113 (+ 10064us) server_negotiation.cc:407] Connection header received
0806 08:57:32.448144 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.448152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.448263 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.448344 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:32.450981 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.452305 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.457593 (+ 5288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.458399 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.458522 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.464640 (+ 6118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:32.464657 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:32.464678 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:32.464738 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:32.469708 (+ 4970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.470782 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.470794 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.470798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.471025 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.471579 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.471584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.471587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.472091 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:32.472478 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.473133 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.473435 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 08:57:32.473832 28919 catalog_manager.cc:4841] TS f582980cffaf4648be4f5c0708bcc096 (127.22.98.1:34199): tablet 081b907659db4ad4bb60d20603172dcc (table test-table [id=f414d32b1d08421b8a7b62ebed952400]) successfully deleted
I20260806 08:57:32.475472 29207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:32.573454 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29021
I20260806 08:57:32.596396 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29227
Aug 06 08:57:32 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:32.613222 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.597225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.22.98.2:38389)
0806 08:57:32.597437 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.597444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.597529 (+ 85us) server_negotiation.cc:407] Connection header received
0806 08:57:32.597782 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.597788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.597917 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.598032 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:32.599734 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.600677 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.602021 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.602422 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.602534 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.606677 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:32.606695 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:32.606714 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:32.606751 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:32.610074 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.610898 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.610907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.610909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.611067 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.611474 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.611478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.611480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.611989 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:32.612283 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.612628 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.612880 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 08:57:32.613958 29399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:32.705919 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29227
I20260806 08:57:32.727988 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29418
Aug 06 08:57:32 dist-test-slave-jpj1 krb5kdc[28880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:32.744207 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:32.728900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.22.98.3:35341)
0806 08:57:32.729126 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:32.729132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:32.729152 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:57:32.729225 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:32.729230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:32.729293 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:32.729380 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:32.730703 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.731455 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.732734 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:32.733142 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:32.733266 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:32.738067 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:32.738086 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:32.738105 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:32.738156 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:32.741194 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.742009 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.742018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.742020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.742177 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:32.742575 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:32.742579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:32.742581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:32.742978 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:32.743252 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:32.743637 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 08:57:32.743896 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 08:57:32.744890 29586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:32.833366 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29418
I20260806 08:57:32.858076 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 28893
I20260806 08:57:32.859310 28982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:32.968573 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 28893
2026-08-06T08:57:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4374 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T08:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:33Z Disabled control of system clock
I20260806 08:57:33.047223 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43261
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:43261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 08:57:33.389472 29666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:33.389822 29666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:33.389945 29666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:33.397393 29666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:33.397519 29666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:33.397584 29666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:33.397722 29666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:33.397778 29666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:33.409469 29666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:43261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:33.410856 29666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:33.412803 29666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:33.424909 29674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:33.424882 29675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:33.424844 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:33.425819 29666 server_base.cc:1034] running on GCE node
I20260806 08:57:33.426712 29666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:33.428298 29666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:33.429538 29666 hybrid_clock.cc:648] HybridClock initialized: now 1786006653429446 us; error 82 us; skew 500 ppm
I20260806 08:57:33.429960 29666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:33.432369 29666 webserver.cc:460] Webserver started at http://127.22.98.62:44251/ using document root <none> and password file <none>
I20260806 08:57:33.433084 29666 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:33.433192 29666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:33.433516 29666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:33.436548 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "5b4e5a3a083647f682e317142e9fcc0a"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "13d4744884e569ab18d6e9c40f40fa34"
server_key_iv: "1379a9f00035074204c507363008df0f"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.437343 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "5b4e5a3a083647f682e317142e9fcc0a"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "13d4744884e569ab18d6e9c40f40fa34"
server_key_iv: "1379a9f00035074204c507363008df0f"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.442610 29666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:33.446148 29688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:33.447522 29666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:33.447751 29666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "5b4e5a3a083647f682e317142e9fcc0a"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "13d4744884e569ab18d6e9c40f40fa34"
server_key_iv: "1379a9f00035074204c507363008df0f"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.447954 29666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:33.453791 29666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:33.454564 29666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:33.454854 29666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:33.473832 29666 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:43261
I20260806 08:57:33.473855 29761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:43261 every 8 connection(s)
I20260806 08:57:33.475203 29666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:33.479024 29762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:33.479913 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29666
I20260806 08:57:33.480085 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:33.480743 22920 external_mini_cluster.cc:1428] Setting key 39fe5e62aecf438132fcc3ee256ad01e
I20260806 08:57:33.489961 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:33.482908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.22.98.62:43261)
0806 08:57:33.484080 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:33.484094 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:33.484152 (+ 58us) server_negotiation.cc:407] Connection header received
0806 08:57:33.485784 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:33.485822 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:33.486402 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:33.486874 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:33.487878 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:33.487886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:33.487903 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:33.487920 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:33.487999 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:33.488020 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:33.488240 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:33.488385 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 08:57:33.488670 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":348,"threads_started":1}
I20260806 08:57:33.493893 29762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: Bootstrap starting.
I20260806 08:57:33.496865 29762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:33.497979 29762 log.cc:824] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:33.500839 29762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: No bootstrap required, opened a new log
I20260806 08:57:33.508251 29762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } }
I20260806 08:57:33.508602 29762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:33.508679 29762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4e5a3a083647f682e317142e9fcc0a, State: Initialized, Role: FOLLOWER
I20260806 08:57:33.509264 29762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } }
I20260806 08:57:33.509479 29762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:33.509593 29762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:33.509728 29762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:33.511965 29762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } }
I20260806 08:57:33.512482 29762 leader_election.cc:302] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b4e5a3a083647f682e317142e9fcc0a; no voters:
I20260806 08:57:33.513218 29762 leader_election.cc:288] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:33.513314 29769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:33.514448 29769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [term 1 LEADER]: Becoming Leader. State: Replica: 5b4e5a3a083647f682e317142e9fcc0a, State: Running, Role: LEADER
I20260806 08:57:33.514974 29769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } }
I20260806 08:57:33.515318 29762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:33.518204 29770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b4e5a3a083647f682e317142e9fcc0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } } }
I20260806 08:57:33.518224 29772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b4e5a3a083647f682e317142e9fcc0a. Latest consensus state: current_term: 1 leader_uuid: "5b4e5a3a083647f682e317142e9fcc0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4e5a3a083647f682e317142e9fcc0a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43261 } } }
I20260806 08:57:33.518689 29772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:33.518695 29770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:33.520478 29779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:33.525295 29779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:33.533840 29779 catalog_manager.cc:1347] Generated new cluster ID: dc315850cd7747918ea5c148477a7194
I20260806 08:57:33.533973 29779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:33.544090 29779 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:33.544625 29779 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:33.559487 29779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: Generated new TSK 0
I20260806 08:57:33.560626 29779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:57:33.568580 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--builtin_ntp_servers=127.22.98.20:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 08:57:33.907105 29795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:33.907392 29795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:33.907509 29795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:33.914669 29795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:33.914770 29795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:33.914922 29795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:33.924121 29795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:33.925376 29795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:33.927152 29795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:33.939909 29803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:33.939946 29804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:33.939973 29806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:33.941039 29795 server_base.cc:1034] running on GCE node
I20260806 08:57:33.941679 29795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:33.942657 29795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:33.943872 29795 hybrid_clock.cc:648] HybridClock initialized: now 1786006653943820 us; error 42 us; skew 500 ppm
I20260806 08:57:33.944197 29795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:33.946398 29795 webserver.cc:460] Webserver started at http://127.22.98.1:40005/ using document root <none> and password file <none>
I20260806 08:57:33.947075 29795 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:33.947186 29795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:33.947510 29795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:33.950351 29795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "ca3b719e60254080a88c79043dd45f6b"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "70e9b9e009500211ecea0ca7eddf62a1"
server_key_iv: "0ce74575fd5ef2eacd736474e6dd69e2"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.951104 29795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "ca3b719e60254080a88c79043dd45f6b"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "70e9b9e009500211ecea0ca7eddf62a1"
server_key_iv: "0ce74575fd5ef2eacd736474e6dd69e2"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.956498 29795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 08:57:33.959899 29813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:33.961203 29795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:57:33.961356 29795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "ca3b719e60254080a88c79043dd45f6b"
format_stamp: "Formatted at 2026-08-06 08:57:33 on dist-test-slave-jpj1"
server_key: "70e9b9e009500211ecea0ca7eddf62a1"
server_key_iv: "0ce74575fd5ef2eacd736474e6dd69e2"
server_key_version: "encryptionkey@0"
I20260806 08:57:33.961547 29795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:33.967074 29795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:33.967891 29795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:33.968142 29795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:33.969024 29795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:33.970415 29795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:33.970510 29795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:33.970620 29795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:33.970691 29795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.002528 29795 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:42783
I20260806 08:57:34.002574 29967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:42783 every 8 connection(s)
I20260806 08:57:34.003938 29795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:34.012391 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.006611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:42427 (local address 127.22.98.62:43261)
0806 08:57:34.007112 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.007119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.008445 (+ 1326us) server_negotiation.cc:407] Connection header received
0806 08:57:34.010154 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.010159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.010216 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.010331 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.011699 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.011704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.011707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.011725 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.011775 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.011787 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.011886 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.012040 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.012094 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":236,"threads_started":1}
I20260806 08:57:34.012707 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29795
I20260806 08:57:34.012897 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:34.013383 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.007247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43261 (local address 127.22.98.1:42427)
0806 08:57:34.008281 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.008327 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.009873 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.010593 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.010602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.010705 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.011358 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.011361 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.011403 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.011496 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.011868 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.011876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.011881 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.012117 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.012332 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":328,"threads_started":1}
I20260806 08:57:34.013547 22920 external_mini_cluster.cc:1428] Setting key 5ac393ca237a283bc6c0268dc7f5488b
I20260806 08:57:34.015251 29968 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43261
I20260806 08:57:34.015672 29968 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:34.016407 29968 heartbeater.cc:511] Master 127.22.98.62:43261 requested a full tablet report, sending...
I20260806 08:57:34.016964 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--builtin_ntp_servers=127.22.98.20:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 08:57:34.018725 29710 ts_manager.cc:194] Registered new tserver with Master: ca3b719e60254080a88c79043dd45f6b (127.22.98.1:42783)
W20260806 08:57:34.354882 29973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:34.355192 29973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:34.355312 29973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:34.362649 29973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:34.362761 29973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:34.362911 29973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:34.372176 29973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:34.373447 29973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:34.375272 29973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:34.388485 29984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:34.388556 29986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:34.388485 29980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:34.389374 29973 server_base.cc:1034] running on GCE node
I20260806 08:57:34.390003 29973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:34.390952 29973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:34.392143 29973 hybrid_clock.cc:648] HybridClock initialized: now 1786006654392081 us; error 55 us; skew 500 ppm
I20260806 08:57:34.392457 29973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:34.394680 29973 webserver.cc:460] Webserver started at http://127.22.98.2:33403/ using document root <none> and password file <none>
I20260806 08:57:34.395375 29973 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:34.395478 29973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:34.395848 29973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:34.398700 29973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "33dfee5a2ce54a3089a96fc4d6529717"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "3ae7cacb814daba63f1e4fd782248abc"
server_key_iv: "d54b54dc89e5d17d4efe1d7686f46fc7"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.399515 29973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "33dfee5a2ce54a3089a96fc4d6529717"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "3ae7cacb814daba63f1e4fd782248abc"
server_key_iv: "d54b54dc89e5d17d4efe1d7686f46fc7"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.404788 29973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:57:34.408268 29997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.409586 29973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 08:57:34.409737 29973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "33dfee5a2ce54a3089a96fc4d6529717"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "3ae7cacb814daba63f1e4fd782248abc"
server_key_iv: "d54b54dc89e5d17d4efe1d7686f46fc7"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.409958 29973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:34.415689 29973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:34.416481 29973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:34.416723 29973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:34.417629 29973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:34.419085 29973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:34.419171 29973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.419255 29973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:34.419324 29973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.450383 29973 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:34157
I20260806 08:57:34.450435 30154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:34157 every 8 connection(s)
I20260806 08:57:34.451745 29973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:34.460086 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.454368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:53075 (local address 127.22.98.62:43261)
0806 08:57:34.454616 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.454622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.456103 (+ 1481us) server_negotiation.cc:407] Connection header received
0806 08:57:34.457819 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.457823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.457894 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.457984 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.459369 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.459373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.459376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.459394 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.459426 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.459438 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.459554 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.459807 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.459860 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 08:57:34.460582 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 29973
I20260806 08:57:34.460721 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:34.461105 30157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.454924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43261 (local address 127.22.98.2:53075)
0806 08:57:34.455942 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.455993 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.457544 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.458263 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.458271 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.458369 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.459055 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.459058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.459099 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.459199 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.459581 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.459592 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.459599 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.459849 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.460058 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":300,"threads_started":1}
I20260806 08:57:34.461179 22920 external_mini_cluster.cc:1428] Setting key 10cde0e1ab67818c153465fda80ea096
I20260806 08:57:34.462545 30155 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43261
I20260806 08:57:34.462841 30155 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:34.463590 30155 heartbeater.cc:511] Master 127.22.98.62:43261 requested a full tablet report, sending...
I20260806 08:57:34.464049 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--builtin_ntp_servers=127.22.98.20:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 08:57:34.465193 29710 ts_manager.cc:194] Registered new tserver with Master: 33dfee5a2ce54a3089a96fc4d6529717 (127.22.98.2:34157)
W20260806 08:57:34.798557 30159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:34.798887 30159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:34.798983 30159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:34.807199 30159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:34.807337 30159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:34.807500 30159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:34.819257 30159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:34.820681 30159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:34.822522 30159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:34.835402 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:34.835583 30171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:34.835557 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:34.836654 30159 server_base.cc:1034] running on GCE node
I20260806 08:57:34.837248 30159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:34.838145 30159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:34.839358 30159 hybrid_clock.cc:648] HybridClock initialized: now 1786006654839319 us; error 38 us; skew 500 ppm
I20260806 08:57:34.839710 30159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:34.841923 30159 webserver.cc:460] Webserver started at http://127.22.98.3:39655/ using document root <none> and password file <none>
I20260806 08:57:34.842604 30159 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:34.842713 30159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:34.843040 30159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:34.845938 30159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "bfc9b8574493407dae83201900e9e941"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "d1f11bf67f5f597ec88b060a01664b94"
server_key_iv: "0f0c2fb381df2f1a327031c0200855e5"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.846716 30159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "bfc9b8574493407dae83201900e9e941"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "d1f11bf67f5f597ec88b060a01664b94"
server_key_iv: "0f0c2fb381df2f1a327031c0200855e5"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.852085 30159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:34.856272 30178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.858371 30159 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 08:57:34.858696 30159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "bfc9b8574493407dae83201900e9e941"
format_stamp: "Formatted at 2026-08-06 08:57:34 on dist-test-slave-jpj1"
server_key: "d1f11bf67f5f597ec88b060a01664b94"
server_key_iv: "0f0c2fb381df2f1a327031c0200855e5"
server_key_version: "encryptionkey@0"
I20260806 08:57:34.859040 30159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:34.867043 30159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:34.868172 30159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:34.868482 30159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:34.869606 30159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:34.871192 30159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:34.871295 30159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.871419 30159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:34.871492 30159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:34.902195 30159 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:43151
I20260806 08:57:34.902240 30330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:43151 every 8 connection(s)
I20260806 08:57:34.903498 30159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:34.906986 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 30159
I20260806 08:57:34.907133 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:34.907693 22920 external_mini_cluster.cc:1428] Setting key fbdb31dc55757354e2a12c202b4c61be
I20260806 08:57:34.912163 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.906167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:55767 (local address 127.22.98.62:43261)
0806 08:57:34.906382 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.906388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.908058 (+ 1670us) server_negotiation.cc:407] Connection header received
0806 08:57:34.909917 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.909923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.909987 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.910102 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.911483 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.911488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.911491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.911508 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.911566 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.911582 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.911694 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.911882 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.911933 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 08:57:34.913062 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.906744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43261 (local address 127.22.98.3:55767)
0806 08:57:34.907878 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.907925 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.909668 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.910352 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.910360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.910465 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.911158 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.911160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.911198 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.911285 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.911671 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.911681 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.911689 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.911957 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.912182 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":334,"threads_started":1}
I20260806 08:57:34.914397 30332 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43261
I20260806 08:57:34.914705 30332 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:34.915336 30332 heartbeater.cc:511] Master 127.22.98.62:43261 requested a full tablet report, sending...
I20260806 08:57:34.916767 29710 ts_manager.cc:194] Registered new tserver with Master: bfc9b8574493407dae83201900e9e941 (127.22.98.3:43151)
I20260806 08:57:34.922757 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 08:57:34.929597 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.927293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.22.98.62:43261)
0806 08:57:34.927513 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.927521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.927885 (+ 364us) server_negotiation.cc:407] Connection header received
0806 08:57:34.928031 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.928035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.928089 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.928161 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.929000 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.929005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.929008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.929022 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.929054 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.929066 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.929158 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.929267 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.929313 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 08:57:34.933502 29710 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:34.940284 29710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:34.973794 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.952548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:34157 (local address 127.0.0.1:49106)
0806 08:57:34.969110 (+ 16562us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.969175 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.969369 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.971191 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.971201 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.971314 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.972274 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.972279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.972316 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.972414 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.972978 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.972990 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.973000 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.973199 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.973298 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16229,"threads_started":1}
I20260806 08:57:34.974658 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.952760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.22.98.2:34157)
0806 08:57:34.968165 (+ 15405us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.968177 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.969527 (+ 1350us) server_negotiation.cc:407] Connection header received
0806 08:57:34.969640 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.969650 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.969944 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.970101 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.972625 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.972637 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.972655 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.972677 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.972761 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.972785 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.972922 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.974202 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.974330 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15184,"thread_start_us":15437,"threads_started":1}
I20260806 08:57:34.978981 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.953022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:43151 (local address 127.0.0.1:34064)
0806 08:57:34.975925 (+ 22903us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.975942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.976039 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.977082 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.977087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.977111 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.977888 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.977891 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.977908 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.977955 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.978369 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.978374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.978379 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.978530 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.978583 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22773,"thread_start_us":22594,"threads_started":1}
I20260806 08:57:34.979040 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.953588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.22.98.3:43151)
0806 08:57:34.974971 (+ 21383us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.974981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.976056 (+ 1075us) server_negotiation.cc:407] Connection header received
0806 08:57:34.976160 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.976170 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.976418 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.976602 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.978172 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.978179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.978190 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.978202 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.978243 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.978267 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.978396 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.978547 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.978649 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21188,"thread_start_us":21403,"threads_started":1}
I20260806 08:57:34.979485 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.952763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59860)
0806 08:57:34.975243 (+ 22480us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:34.975266 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:34.975350 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:34.977302 (+ 1952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:34.977307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:34.977332 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:34.978523 (+ 1191us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:34.978526 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:34.978543 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:34.978587 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:34.979078 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:34.979083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:34.979088 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:57:34.979183 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 08:57:34.979232 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22379,"thread_start_us":15831,"threads_started":1}
I20260806 08:57:34.980369 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:34.953332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.22.98.1:42783)
0806 08:57:34.975626 (+ 22294us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:34.975634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:34.975666 (+ 32us) server_negotiation.cc:407] Connection header received
0806 08:57:34.975736 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:34.975743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:34.976463 (+ 720us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:34.976615 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:34.978759 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:34.978768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:34.978782 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:34.978810 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:34.978868 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:34.978888 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:34.979017 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:34.979951 (+ 934us) server_negotiation.cc:299] Negotiation successful
0806 08:57:34.980041 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22104,"thread_start_us":21953,"threads_started":1}
I20260806 08:57:34.980266 30058 tablet_service.cc:1467] Processing CreateTablet for tablet 7dee6e1177744c549ceafa534036a0da (DEFAULT_TABLE table=test-table [id=e2ecbb5b98fd40cd89cc15f5df0b3940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:34.981899 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dee6e1177744c549ceafa534036a0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:34.982916 30237 tablet_service.cc:1467] Processing CreateTablet for tablet 7dee6e1177744c549ceafa534036a0da (DEFAULT_TABLE table=test-table [id=e2ecbb5b98fd40cd89cc15f5df0b3940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:34.984122 30237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dee6e1177744c549ceafa534036a0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:34.983870 29878 tablet_service.cc:1467] Processing CreateTablet for tablet 7dee6e1177744c549ceafa534036a0da (DEFAULT_TABLE table=test-table [id=e2ecbb5b98fd40cd89cc15f5df0b3940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:34.985464 29878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dee6e1177744c549ceafa534036a0da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:34.995034 30361 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Bootstrap starting.
I20260806 08:57:34.997120 30362 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Bootstrap starting.
I20260806 08:57:34.999056 30361 tablet_bootstrap.cc:654] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:35.000398 30363 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: Bootstrap starting.
I20260806 08:57:35.000620 30361 log.cc:824] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:35.000717 30362 tablet_bootstrap.cc:654] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:35.001983 30362 log.cc:824] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:35.004284 30361 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: No bootstrap required, opened a new log
I20260806 08:57:35.004595 30361 ts_tablet_manager.cc:1397] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 08:57:35.004805 30363 tablet_bootstrap.cc:654] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:35.006338 30363 log.cc:824] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:35.011639 30362 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: No bootstrap required, opened a new log
I20260806 08:57:35.011998 30362 ts_tablet_manager.cc:1397] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260806 08:57:35.017465 30361 raft_consensus.cc:348] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.017941 30361 raft_consensus.cc:374] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:35.018049 30361 raft_consensus.cc:729] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc9b8574493407dae83201900e9e941, State: Initialized, Role: FOLLOWER
I20260806 08:57:35.018795 30361 consensus_queue.cc:260] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.020687 30332 heartbeater.cc:503] Master 127.22.98.62:43261 was elected leader, sending a full tablet report...
I20260806 08:57:35.021360 30361 ts_tablet_manager.cc:1428] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260806 08:57:35.022033 29968 heartbeater.cc:503] Master 127.22.98.62:43261 was elected leader, sending a full tablet report...
I20260806 08:57:35.022296 30363 tablet_bootstrap.cc:492] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: No bootstrap required, opened a new log
I20260806 08:57:35.022620 30363 ts_tablet_manager.cc:1397] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.005s
I20260806 08:57:35.023070 30362 raft_consensus.cc:348] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.023581 30362 raft_consensus.cc:374] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:35.023682 30362 raft_consensus.cc:729] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33dfee5a2ce54a3089a96fc4d6529717, State: Initialized, Role: FOLLOWER
I20260806 08:57:35.024526 30362 consensus_queue.cc:260] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.028232 30155 heartbeater.cc:503] Master 127.22.98.62:43261 was elected leader, sending a full tablet report...
I20260806 08:57:35.029011 30362 ts_tablet_manager.cc:1428] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Time spent starting tablet: real 0.017s user 0.009s sys 0.006s
I20260806 08:57:35.032482 30363 raft_consensus.cc:348] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.032953 30363 raft_consensus.cc:374] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:35.033051 30363 raft_consensus.cc:729] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3b719e60254080a88c79043dd45f6b, State: Initialized, Role: FOLLOWER
I20260806 08:57:35.033609 30363 consensus_queue.cc:260] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.035099 30363 ts_tablet_manager.cc:1428] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260806 08:57:35.155279 30333 tablet.cc:2369] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:35.181660 30368 raft_consensus.cc:482] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:35.182124 30368 raft_consensus.cc:504] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.184033 30368 leader_election.cc:288] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33dfee5a2ce54a3089a96fc4d6529717 (127.22.98.2:34157), ca3b719e60254080a88c79043dd45f6b (127.22.98.1:42783)
I20260806 08:57:35.186965 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.184562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:34157 (local address 127.22.98.3:44505)
0806 08:57:35.184834 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.184862 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:35.185000 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:35.185387 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:35.185390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:35.185411 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:35.186059 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:35.186061 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:35.186075 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:35.186135 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:35.186512 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:35.186518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:35.186523 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:57:35.186647 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 08:57:35.186703 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 08:57:35.186976 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.184494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:44505 (local address 127.22.98.2:34157)
0806 08:57:35.184684 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.184690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.184959 (+ 269us) server_negotiation.cc:407] Connection header received
0806 08:57:35.185143 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.185148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.185215 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.185333 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.186295 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.186300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.186303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.186320 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.186354 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.186370 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.186497 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.186671 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.186729 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 08:57:35.187350 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.184612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:60895 (local address 127.22.98.1:42783)
0806 08:57:35.184866 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.184870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.185457 (+ 587us) server_negotiation.cc:407] Connection header received
0806 08:57:35.185588 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.185639 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.185708 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.185786 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.186710 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.186715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.186720 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.186739 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.186778 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.186794 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.186900 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.187022 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.187083 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 08:57:35.187527 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.184632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.22.98.3:60895)
0806 08:57:35.185352 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.185368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:35.185479 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:35.185867 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:35.185873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:35.185898 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:35.186508 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:57:35.186512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:35.186528 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:57:35.186565 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:35.186889 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:35.186893 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:35.186898 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:57:35.187039 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 08:57:35.187091 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":393,"threads_started":1}
I20260806 08:57:35.187964 30082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dee6e1177744c549ceafa534036a0da" candidate_uuid: "bfc9b8574493407dae83201900e9e941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33dfee5a2ce54a3089a96fc4d6529717" is_pre_election: true
I20260806 08:57:35.188638 30082 raft_consensus.cc:2454] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc9b8574493407dae83201900e9e941 in term 0.
I20260806 08:57:35.188572 29908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dee6e1177744c549ceafa534036a0da" candidate_uuid: "bfc9b8574493407dae83201900e9e941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3b719e60254080a88c79043dd45f6b" is_pre_election: true
I20260806 08:57:35.189237 29908 raft_consensus.cc:2454] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfc9b8574493407dae83201900e9e941 in term 0.
I20260806 08:57:35.189402 30182 leader_election.cc:302] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33dfee5a2ce54a3089a96fc4d6529717, bfc9b8574493407dae83201900e9e941; no voters:
I20260806 08:57:35.189831 30368 raft_consensus.cc:2781] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:35.189967 30368 raft_consensus.cc:482] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:35.190064 30368 raft_consensus.cc:3037] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:35.192361 30368 raft_consensus.cc:504] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.192950 30368 leader_election.cc:288] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [CANDIDATE]: Term 1 election: Requested vote from peers 33dfee5a2ce54a3089a96fc4d6529717 (127.22.98.2:34157), ca3b719e60254080a88c79043dd45f6b (127.22.98.1:42783)
I20260806 08:57:35.193473 30082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dee6e1177744c549ceafa534036a0da" candidate_uuid: "bfc9b8574493407dae83201900e9e941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33dfee5a2ce54a3089a96fc4d6529717"
I20260806 08:57:35.193472 29908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dee6e1177744c549ceafa534036a0da" candidate_uuid: "bfc9b8574493407dae83201900e9e941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3b719e60254080a88c79043dd45f6b"
I20260806 08:57:35.193670 30082 raft_consensus.cc:3037] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:35.193714 29908 raft_consensus.cc:3037] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:35.196077 30082 raft_consensus.cc:2454] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc9b8574493407dae83201900e9e941 in term 1.
I20260806 08:57:35.196084 29908 raft_consensus.cc:2454] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfc9b8574493407dae83201900e9e941 in term 1.
I20260806 08:57:35.196465 30182 leader_election.cc:302] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33dfee5a2ce54a3089a96fc4d6529717, bfc9b8574493407dae83201900e9e941; no voters:
I20260806 08:57:35.196781 30368 raft_consensus.cc:2781] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:35.197229 30368 raft_consensus.cc:686] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 1 LEADER]: Becoming Leader. State: Replica: bfc9b8574493407dae83201900e9e941, State: Running, Role: LEADER
I20260806 08:57:35.197784 30368 consensus_queue.cc:237] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } }
I20260806 08:57:35.202536 29711 catalog_manager.cc:5495] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfc9b8574493407dae83201900e9e941 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "bfc9b8574493407dae83201900e9e941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc9b8574493407dae83201900e9e941" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 43151 } health_report { overall_health: HEALTHY } } }
W20260806 08:57:35.203740 30156 tablet.cc:2369] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:35.215574 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.213389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.22.98.3:43151)
0806 08:57:35.213576 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.213582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.213602 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:57:35.213802 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.213807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.213922 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.214013 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.214911 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.214918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.214923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.214941 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.215000 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.215018 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.215133 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.215250 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.215304 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 08:57:35.224323 30082 raft_consensus.cc:1261] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 1 FOLLOWER]: Refusing update from remote peer bfc9b8574493407dae83201900e9e941: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:35.224332 29908 raft_consensus.cc:1261] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 1 FOLLOWER]: Refusing update from remote peer bfc9b8574493407dae83201900e9e941: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:35.225317 30373 consensus_queue.cc:1035] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33dfee5a2ce54a3089a96fc4d6529717" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:35.225700 30368 consensus_queue.cc:1035] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca3b719e60254080a88c79043dd45f6b" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:35.234983 30377 mvcc.cc:205] Tried to move back new op lower bound from 7315483259788005376 to 7315483259694252032. Current Snapshot: MvccSnapshot[applied={T|T < 7315483259788005376}]
I20260806 08:57:35.254117 29710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:35.254413 29710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:35.259119 29710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b4e5a3a083647f682e317142e9fcc0a: Sending DeleteTablet for 3 replicas of tablet 7dee6e1177744c549ceafa534036a0da
I20260806 08:57:35.260484 29878 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dee6e1177744c549ceafa534036a0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:35 UTC) from {username='slave'} at 127.0.0.1:59860
I20260806 08:57:35.261035 30237 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dee6e1177744c549ceafa534036a0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:35 UTC) from {username='slave'} at 127.0.0.1:34064
I20260806 08:57:35.261168 30058 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dee6e1177744c549ceafa534036a0da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:35 UTC) from {username='slave'} at 127.0.0.1:49106
I20260806 08:57:35.264144 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29795
I20260806 08:57:35.272940 30392 tablet_replica.cc:330] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: stopping tablet replica
I20260806 08:57:35.273468 30392 raft_consensus.cc:2229] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:35.273797 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.265718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.22.98.1:42783)
0806 08:57:35.265904 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.265911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.270245 (+ 4334us) server_negotiation.cc:407] Connection header received
0806 08:57:35.271197 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.271203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.271333 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.271411 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.272965 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.272971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.272975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.272998 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.273036 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.273053 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.273149 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.273412 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.273479 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 08:57:35.274230 30392 raft_consensus.cc:2258] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:35.274900 29962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:35.274969 30390 tablet_replica.cc:330] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b: stopping tablet replica
I20260806 08:57:35.277415 30392 ts_tablet_manager.cc:1905] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:35.277573 30394 tablet_replica.cc:330] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: stopping tablet replica
I20260806 08:57:35.278007 30394 raft_consensus.cc:2229] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:35.278523 30394 raft_consensus.cc:2258] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:35.282163 30394 ts_tablet_manager.cc:1905] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:35.284945 30392 ts_tablet_manager.cc:1918] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:35.285143 30392 log.cc:1196] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/7dee6e1177744c549ceafa534036a0da
I20260806 08:57:35.285732 30392 ts_tablet_manager.cc:1939] T 7dee6e1177744c549ceafa534036a0da P bfc9b8574493407dae83201900e9e941: Deleting consensus metadata
I20260806 08:57:35.287711 29690 catalog_manager.cc:4841] TS bfc9b8574493407dae83201900e9e941 (127.22.98.3:43151): tablet 7dee6e1177744c549ceafa534036a0da (table test-table [id=e2ecbb5b98fd40cd89cc15f5df0b3940]) successfully deleted
I20260806 08:57:35.288791 30394 ts_tablet_manager.cc:1918] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:35.288940 30394 log.cc:1196] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/7dee6e1177744c549ceafa534036a0da
I20260806 08:57:35.289355 30394 ts_tablet_manager.cc:1939] T 7dee6e1177744c549ceafa534036a0da P 33dfee5a2ce54a3089a96fc4d6529717: Deleting consensus metadata
I20260806 08:57:35.290836 29692 catalog_manager.cc:4841] TS 33dfee5a2ce54a3089a96fc4d6529717 (127.22.98.2:34157): tablet 7dee6e1177744c549ceafa534036a0da (table test-table [id=e2ecbb5b98fd40cd89cc15f5df0b3940]) successfully deleted
I20260806 08:57:35.275388 30390 raft_consensus.cc:2229] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:35.377043 30390 raft_consensus.cc:2258] T 7dee6e1177744c549ceafa534036a0da P ca3b719e60254080a88c79043dd45f6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:35.377254 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29795
I20260806 08:57:35.401221 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29973
W20260806 08:57:35.401079 29689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:42783 (error 108)
I20260806 08:57:35.405856 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.405323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59878)
0806 08:57:35.405474 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.405684 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 08:57:35.405869 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.402721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.22.98.2:34157)
0806 08:57:35.402887 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.402893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.402909 (+ 16us) server_negotiation.cc:407] Connection header received
0806 08:57:35.402969 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.402975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.403079 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.403191 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.404347 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.404354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.404358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.404378 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.404415 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.404430 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.404550 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.404861 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.404960 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 08:57:35.406279 29689 catalog_manager.cc:4584] TS ca3b719e60254080a88c79043dd45f6b (127.22.98.1:42783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7dee6e1177744c549ceafa534036a0da: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
I20260806 08:57:35.406567 30146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:35.460233 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.459743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59890)
0806 08:57:35.459943 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.460032 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 08:57:35.461261 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.460888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59902)
0806 08:57:35.461036 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.461123 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 08:57:35.521787 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.521246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59910)
0806 08:57:35.521463 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.521560 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 08:57:35.522923 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.522549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59922)
0806 08:57:35.522728 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.522801 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 08:57:35.554487 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29973
I20260806 08:57:35.574680 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 30159
I20260806 08:57:35.577913 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.575453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.22.98.3:43151)
0806 08:57:35.575648 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:35.575654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:35.575705 (+ 51us) server_negotiation.cc:407] Connection header received
0806 08:57:35.575981 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:35.575986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:35.576043 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:35.576156 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:35.577184 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:35.577192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:35.577196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:35.577216 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:35.577251 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:35.577268 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:35.577403 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:35.577513 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 08:57:35.577562 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 08:57:35.578634 30325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:35.603919 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.603352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59928)
0806 08:57:35.603651 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.603753 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 08:57:35.605048 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.604620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59942)
0806 08:57:35.604831 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.604904 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 08:57:35.701347 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 30159
I20260806 08:57:35.728190 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 29666
I20260806 08:57:35.729457 29755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:35.838392 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:35.837851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42783 (local address 127.0.0.1:59946)
0806 08:57:35.838067 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:35.838208 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 08:57:35.838584 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 29666
2026-08-06T08:57:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2868 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 08:57:35.883962 22920 mini_oidc.cc:152] Starting JWKS server
I20260806 08:57:35.886909 22920 webserver.cc:460] Webserver started at https://127.0.0.1:45095/ using document root <none> and password file <none>
I20260806 08:57:35.887280 22920 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 08:57:35.888051 22920 webserver.cc:460] Webserver started at http://127.0.0.1:44707/ using document root <none> and password file <none>
2026-08-06T08:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:35Z Disabled control of system clock
I20260806 08:57:35.924304 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39031
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:39031
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45095/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/testchainca.cert with env {}
W20260806 08:57:36.273566 30412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:36.273922 30412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:36.274058 30412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:36.281596 30412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:36.281744 30412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:36.281821 30412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:36.281905 30412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:36.281966 30412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 08:57:36.282058 30412 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 08:57:36.282111 30412 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45095/jwks.json
I20260806 08:57:36.292722 30412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:39031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45095/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:36.294160 30412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:36.295564 30412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:36.308100 30420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:36.308166 30418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:36.308144 30417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:36.309363 30412 server_base.cc:1034] running on GCE node
I20260806 08:57:36.310273 30412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:36.311880 30412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:36.313133 30412 hybrid_clock.cc:648] HybridClock initialized: now 1786006656313065 us; error 47 us; skew 500 ppm
I20260806 08:57:36.313526 30412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:36.316152 30412 webserver.cc:460] Webserver started at http://127.22.98.62:45863/ using document root <none> and password file <none>
I20260806 08:57:36.316955 30412 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:36.317088 30412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:36.317430 30412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:36.320588 30412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "16d5da51fec647dabb369795e757e839"
format_stamp: "Formatted at 2026-08-06 08:57:36 on dist-test-slave-jpj1"
server_key: "48551dc5ab44aa21bed8626f795b5132"
server_key_iv: "0acea5ec0cae37582b5cdc8fbb59b275"
server_key_version: "encryptionkey@0"
I20260806 08:57:36.321424 30412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "16d5da51fec647dabb369795e757e839"
format_stamp: "Formatted at 2026-08-06 08:57:36 on dist-test-slave-jpj1"
server_key: "48551dc5ab44aa21bed8626f795b5132"
server_key_iv: "0acea5ec0cae37582b5cdc8fbb59b275"
server_key_version: "encryptionkey@0"
I20260806 08:57:36.326958 30412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:36.330844 30430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:36.332360 30412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:36.332530 30412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "16d5da51fec647dabb369795e757e839"
format_stamp: "Formatted at 2026-08-06 08:57:36 on dist-test-slave-jpj1"
server_key: "48551dc5ab44aa21bed8626f795b5132"
server_key_iv: "0acea5ec0cae37582b5cdc8fbb59b275"
server_key_version: "encryptionkey@0"
I20260806 08:57:36.332746 30412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:36.360872 30412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:36.361799 30412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:36.362131 30412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:36.380410 30412 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:39031
I20260806 08:57:36.380436 30510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:39031 every 8 connection(s)
I20260806 08:57:36.381786 30412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:36.385810 30511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:36.388737 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 30412
I20260806 08:57:36.388959 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:36.389765 22920 external_mini_cluster.cc:1428] Setting key 627f37ef816e800b94f2484553717b18
I20260806 08:57:36.399274 30511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839: Bootstrap starting.
I20260806 08:57:36.403239 30511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:36.404543 30511 log.cc:824] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:36.405162 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:36.391995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.22.98.62:39031)
0806 08:57:36.393098 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:36.393111 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:36.393157 (+ 46us) server_negotiation.cc:407] Connection header received
0806 08:57:36.395000 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:36.395035 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:36.395665 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:36.396142 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:36.396912 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.398287 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.400161 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.400927 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.401139 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:36.402453 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:36.402462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:36.402479 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:36.403110 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:36.403220 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:36.403247 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:36.403352 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:36.403634 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 08:57:36.404014 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":419,"threads_started":1}
I20260806 08:57:36.407970 30511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839: No bootstrap required, opened a new log
I20260806 08:57:36.409747 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 08:57:36.416663 30511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } }
I20260806 08:57:36.417107 30511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:36.417207 30511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d5da51fec647dabb369795e757e839, State: Initialized, Role: FOLLOWER
I20260806 08:57:36.417804 30511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } }
I20260806 08:57:36.418064 30511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:36.418195 30511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:36.418354 30511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:36.421094 30511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } }
I20260806 08:57:36.421639 30511 leader_election.cc:302] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d5da51fec647dabb369795e757e839; no voters:
I20260806 08:57:36.422273 30511 leader_election.cc:288] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:36.422461 30518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:36.423197 30518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [term 1 LEADER]: Becoming Leader. State: Replica: 16d5da51fec647dabb369795e757e839, State: Running, Role: LEADER
I20260806 08:57:36.423794 30518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } }
I20260806 08:57:36.424578 30511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:36.426703 30519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16d5da51fec647dabb369795e757e839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } } }
I20260806 08:57:36.427129 30519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:36.428601 30521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d5da51fec647dabb369795e757e839. Latest consensus state: current_term: 1 leader_uuid: "16d5da51fec647dabb369795e757e839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d5da51fec647dabb369795e757e839" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39031 } } }
I20260806 08:57:36.428853 30521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:36.429373 30524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:36.435119 30524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:36.444600 30524 catalog_manager.cc:1347] Generated new cluster ID: 722e293dd68943798ef878ec4ea972d5
I20260806 08:57:36.444756 30524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:36.467490 30524 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:36.468870 30524 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:36.477420 30524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16d5da51fec647dabb369795e757e839: Generated new TSK 0
I20260806 08:57:36.478338 30524 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 08:57:36.529686 30555 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 08:57:36.529922 30555 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 08:57:36.531347 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:36.508095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.22.98.62:39031)
0806 08:57:36.508312 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:36.508319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:36.508762 (+ 443us) server_negotiation.cc:407] Connection header received
0806 08:57:36.509192 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:36.509198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:36.509301 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:36.509459 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0806 08:57:36.510050 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.511197 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.512731 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.513436 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.513573 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:36.514167 (+ 594us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 08:57:36.530598 (+ 16431us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 08:57:36.531005 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 08:57:36.531474 30514 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 08:57:36.533296 30554 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:36.507911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:39031 (local address 127.0.0.1:52264)
0806 08:57:36.508661 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:36.508677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:36.508845 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:36.509511 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:36.509515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:36.509526 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 08:57:36.509892 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:36.509902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.511369 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.511374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:36.512584 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:36.512598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:36.513607 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:36.513612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:36.513733 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:36.513819 (+ 86us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 08:57:36.531022 (+ 17203us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 08:57:36.531361 (+ 339us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.98.62:39031: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":401,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.98.62:39031: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 08:57:36.535447 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 30412
I20260806 08:57:36.536607 30504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:36.634436 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 30412
2026-08-06T08:57:36Z chronyd exiting
I20260806 08:57:36.690667 22920 test_util.cc:175] -----------------------------------------------
I20260806 08:57:36.690819 22920 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786006610327111-22920-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (811 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 08:57:36.694629 22920 mini_oidc.cc:152] Starting JWKS server
I20260806 08:57:36.696538 22920 webserver.cc:460] Webserver started at https://127.0.0.1:42763/ using document root <none> and password file <none>
I20260806 08:57:36.696744 22920 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 08:57:36.697357 22920 webserver.cc:460] Webserver started at http://127.0.0.1:41339/ using document root <none> and password file <none>
2026-08-06T08:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:36Z Disabled control of system clock
I20260806 08:57:36.731933 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:45871
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:45871
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42763/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/testchainca.cert with env {}
W20260806 08:57:37.064733 30566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:37.065088 30566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:37.065240 30566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:37.072599 30566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:37.072742 30566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:37.072809 30566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:37.072893 30566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:37.072954 30566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 08:57:37.073040 30566 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 08:57:37.073094 30566 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42763/jwks.json
I20260806 08:57:37.083418 30566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:45871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:45871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42763/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:37.084867 30566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:37.086297 30566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:37.099169 30573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:37.099193 30574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:37.099241 30579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:37.100551 30566 server_base.cc:1034] running on GCE node
I20260806 08:57:37.101845 30566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:37.103688 30566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:37.104947 30566 hybrid_clock.cc:648] HybridClock initialized: now 1786006657104895 us; error 40 us; skew 500 ppm
I20260806 08:57:37.105369 30566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:37.112660 30566 webserver.cc:460] Webserver started at http://127.22.98.62:41353/ using document root <none> and password file <none>
I20260806 08:57:37.113492 30566 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:37.113607 30566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:37.114043 30566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:37.117297 30566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "424359a88f614415a4bd500df4ed8385"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "45175c7535043c166022aaa062f803d5"
server_key_iv: "a0d75126b05ada9381efcec3004b3411"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.118150 30566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "424359a88f614415a4bd500df4ed8385"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "45175c7535043c166022aaa062f803d5"
server_key_iv: "a0d75126b05ada9381efcec3004b3411"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.123824 30566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:37.127588 30587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:37.129094 30566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:37.129300 30566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "424359a88f614415a4bd500df4ed8385"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "45175c7535043c166022aaa062f803d5"
server_key_iv: "a0d75126b05ada9381efcec3004b3411"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.129515 30566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:37.159603 30566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:37.160593 30566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:37.160933 30566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:37.178755 30566 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:45871
I20260806 08:57:37.178802 30663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:45871 every 8 connection(s)
I20260806 08:57:37.180188 30566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:37.184088 30664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:37.185293 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 30566
I20260806 08:57:37.185489 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:37.186370 22920 external_mini_cluster.cc:1428] Setting key 6f3d765f1f2e163c4a08808a48d229ff
I20260806 08:57:37.198071 30664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385: Bootstrap starting.
I20260806 08:57:37.200392 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:37.188476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.22.98.62:45871)
0806 08:57:37.189711 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:37.189726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:37.189776 (+ 50us) server_negotiation.cc:407] Connection header received
0806 08:57:37.191406 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:37.191434 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:37.192028 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:37.192471 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:37.193067 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.194351 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.195692 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.196405 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.196595 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:37.197920 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:37.197928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:37.197944 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:37.198553 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:37.198639 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:37.198671 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:37.198795 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:37.199050 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 08:57:37.199366 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":446,"threads_started":1}
I20260806 08:57:37.201511 30664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:37.202603 30664 log.cc:824] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:37.204365 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 08:57:37.205835 30664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385: No bootstrap required, opened a new log
I20260806 08:57:37.214053 30664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } }
I20260806 08:57:37.214493 30664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:37.214591 30664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 424359a88f614415a4bd500df4ed8385, State: Initialized, Role: FOLLOWER
I20260806 08:57:37.215258 30664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } }
I20260806 08:57:37.215523 30664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:37.215718 30664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:37.215895 30664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:37.218173 30664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } }
I20260806 08:57:37.218745 30664 leader_election.cc:302] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 424359a88f614415a4bd500df4ed8385; no voters:
I20260806 08:57:37.219396 30664 leader_election.cc:288] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:37.219607 30674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:37.220340 30674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [term 1 LEADER]: Becoming Leader. State: Replica: 424359a88f614415a4bd500df4ed8385, State: Running, Role: LEADER
I20260806 08:57:37.220986 30674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } }
I20260806 08:57:37.221761 30664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:37.224315 30675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "424359a88f614415a4bd500df4ed8385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } } }
I20260806 08:57:37.224781 30675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:37.224689 30677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 424359a88f614415a4bd500df4ed8385. Latest consensus state: current_term: 1 leader_uuid: "424359a88f614415a4bd500df4ed8385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "424359a88f614415a4bd500df4ed8385" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 45871 } } }
I20260806 08:57:37.225039 30677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:37.228957 30685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:37.235039 30685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:37.245865 30685 catalog_manager.cc:1347] Generated new cluster ID: 72a27cab996d49eba71ae5f3d94b39be
I20260806 08:57:37.245991 30685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:37.267115 30685 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:37.268675 30685 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:37.277773 30685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 424359a88f614415a4bd500df4ed8385: Generated new TSK 0
I20260806 08:57:37.278694 30685 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 08:57:37.312003 30704 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 08:57:37.312148 30704 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 08:57:37.313232 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:37.293056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.22.98.62:45871)
0806 08:57:37.293289 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:37.293296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:37.293727 (+ 431us) server_negotiation.cc:407] Connection header received
0806 08:57:37.293972 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:37.293976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:37.294055 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:37.294178 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0806 08:57:37.294768 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.295497 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.297525 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.297987 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.298110 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:37.298765 (+ 655us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 08:57:37.312620 (+ 13855us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 08:57:37.312994 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60896: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 08:57:37.313370 30671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60896: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 08:57:37.313650 30703 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:37.292872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:45871 (local address 127.0.0.1:60896)
0806 08:57:37.293611 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:37.293636 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:37.293794 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:37.294223 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:37.294229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:37.294243 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 08:57:37.294579 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:37.294592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.295657 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.295663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:37.297381 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:37.297399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.298147 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.298154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:37.298372 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:37.298421 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 08:57:37.312922 (+ 14501us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 08:57:37.313248 (+ 326us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.98.62:45871: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42763/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":421,"threads_started":1}
I20260806 08:57:37.315050 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 30566
I20260806 08:57:37.316243 30656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:37.412945 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 30566
2026-08-06T08:57:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (781 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 08:57:37.476692 22920 mini_oidc.cc:152] Starting JWKS server
I20260806 08:57:37.478596 22920 webserver.cc:460] Webserver started at https://127.0.0.1:36581/ using document root <none> and password file <none>
I20260806 08:57:37.478829 22920 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 08:57:37.479478 22920 webserver.cc:460] Webserver started at http://127.0.0.1:42887/ using document root <none> and password file <none>
2026-08-06T08:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:37Z Disabled control of system clock
I20260806 08:57:37.517369 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37127
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:37175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37127
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36581/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 08:57:37.872874 30716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:37.873207 30716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:37.873354 30716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:37.880776 30716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:37.880908 30716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:37.880978 30716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:37.881062 30716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:37.881125 30716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 08:57:37.881212 30716 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 08:57:37.881274 30716 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36581/jwks.json
I20260806 08:57:37.890847 30716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36581/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:37.892248 30716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:37.894146 30716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 08:57:37.906430 30716 server_base.cc:1034] running on GCE node
W20260806 08:57:37.906481 30730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:37.906401 30726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:37.906817 30725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:37.907775 30716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:37.909214 30716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:37.910454 30716 hybrid_clock.cc:648] HybridClock initialized: now 1786006657910392 us; error 37 us; skew 500 ppm
I20260806 08:57:37.910879 30716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:37.913321 30716 webserver.cc:460] Webserver started at http://127.22.98.62:37641/ using document root <none> and password file <none>
I20260806 08:57:37.914016 30716 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:37.914108 30716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:37.914393 30716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:37.917392 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "9edb6eb1e67f44c3aabd03066f15128d"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "67652d608f4e69fc289c1fd1b51da06b"
server_key_iv: "d2f64f45df7bcca5cda8baeca5a0efa7"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.918164 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "9edb6eb1e67f44c3aabd03066f15128d"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "67652d608f4e69fc289c1fd1b51da06b"
server_key_iv: "d2f64f45df7bcca5cda8baeca5a0efa7"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.923831 30716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:57:37.927771 30737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:37.929359 30716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:37.929553 30716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "9edb6eb1e67f44c3aabd03066f15128d"
format_stamp: "Formatted at 2026-08-06 08:57:37 on dist-test-slave-jpj1"
server_key: "67652d608f4e69fc289c1fd1b51da06b"
server_key_iv: "d2f64f45df7bcca5cda8baeca5a0efa7"
server_key_version: "encryptionkey@0"
I20260806 08:57:37.929833 30716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:37.951812 30716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:37.952693 30716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:37.953017 30716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:37.970961 30716 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37127
I20260806 08:57:37.970980 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37127 every 8 connection(s)
I20260806 08:57:37.972429 30716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:37.976567 30809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:37.982652 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 30716
I20260806 08:57:37.982877 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:37.983515 22920 external_mini_cluster.cc:1428] Setting key 4d4f074aa56443d602b635fb9f378a41
I20260806 08:57:37.988708 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d: Bootstrap starting.
I20260806 08:57:37.993013 30809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:37.994165 30809 log.cc:824] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:37.997713 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d: No bootstrap required, opened a new log
I20260806 08:57:37.999985 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:37.985837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.22.98.62:37127)
0806 08:57:37.986992 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:37.987008 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:37.987067 (+ 59us) server_negotiation.cc:407] Connection header received
0806 08:57:37.989482 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:37.989515 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:37.990182 (+ 667us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:37.990692 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:37.991273 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.993234 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.994674 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:37.995454 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:37.995708 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:37.996864 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:37.996876 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:37.996895 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:37.997649 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:37.997771 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:37.997802 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:37.997914 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:37.998224 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 08:57:37.998597 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":325,"threads_started":1}
I20260806 08:57:38.004555 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 08:57:38.007004 30809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } }
I20260806 08:57:38.007524 30809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:38.007665 30809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edb6eb1e67f44c3aabd03066f15128d, State: Initialized, Role: FOLLOWER
I20260806 08:57:38.008327 30809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } }
I20260806 08:57:38.008546 30809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:38.008653 30809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:38.008778 30809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:38.011005 30809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } }
I20260806 08:57:38.011590 30809 leader_election.cc:302] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9edb6eb1e67f44c3aabd03066f15128d; no voters:
I20260806 08:57:38.012264 30809 leader_election.cc:288] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:38.012409 30819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:38.013139 30819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [term 1 LEADER]: Becoming Leader. State: Replica: 9edb6eb1e67f44c3aabd03066f15128d, State: Running, Role: LEADER
I20260806 08:57:38.013697 30819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } }
I20260806 08:57:38.014467 30809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:38.016495 30825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9edb6eb1e67f44c3aabd03066f15128d. Latest consensus state: current_term: 1 leader_uuid: "9edb6eb1e67f44c3aabd03066f15128d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } } }
I20260806 08:57:38.016844 30825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:38.018136 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9edb6eb1e67f44c3aabd03066f15128d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edb6eb1e67f44c3aabd03066f15128d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37127 } } }
I20260806 08:57:38.018404 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:38.020704 30831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:38.026110 30831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:38.038010 30831 catalog_manager.cc:1347] Generated new cluster ID: dd3ff8c22c2f43d68d3c83cc9cfcabc9
I20260806 08:57:38.038141 30831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:38.056344 30831 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:38.057740 30831 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:38.071739 30831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9edb6eb1e67f44c3aabd03066f15128d: Generated new TSK 0
I20260806 08:57:38.072679 30831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:57:38.097205 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:38.089654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.22.98.62:37127)
0806 08:57:38.089868 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:38.089875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:38.090280 (+ 405us) server_negotiation.cc:407] Connection header received
0806 08:57:38.090524 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:38.090528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:38.090601 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:38.090753 (+ 152us) server_negotiation.cc:226] Negotiated authn=JWT
0806 08:57:38.091260 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.092086 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.093453 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.093886 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.093998 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:38.094520 (+ 522us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 08:57:38.096494 (+ 1974us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 08:57:38.096860 (+ 366us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 08:57:38.097458 30815 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 08:57:38.097965 30859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:38.089470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37127 (local address 127.0.0.1:49130)
0806 08:57:38.090142 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:38.090159 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:38.090337 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:38.090789 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:38.090794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:38.090808 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 08:57:38.091098 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:38.091107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.092235 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.092239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:38.093303 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:38.093315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.094077 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.094085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:38.094233 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:38.094268 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 08:57:38.096704 (+ 2436us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 08:57:38.096945 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.98.62:37127: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":406,"threads_started":1}
I20260806 08:57:38.099298 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 30716
I20260806 08:57:38.100732 30802 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 08:57:38.109833 30861 webserver.cc:533] Webserver: sslize failed:
E20260806 08:57:38.109963 30861 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 08:57:38.196246 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 30716
2026-08-06T08:57:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (784 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T08:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:38Z Disabled control of system clock
I20260806 08:57:38.298159 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41417
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:41417
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/testchainca.cert with env {}
W20260806 08:57:38.642783 30867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:38.643122 30867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:38.643235 30867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:38.650782 30867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:38.650900 30867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:38.650949 30867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:38.651034 30867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:38.651099 30867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 08:57:38.651167 30867 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 08:57:38.651223 30867 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 08:57:38.661592 30867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:41417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:38.663051 30867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:38.664552 30867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:38.677060 30878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:38.677096 30874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:38.677053 30875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:38.677982 30867 server_base.cc:1034] running on GCE node
I20260806 08:57:38.678843 30867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:38.680487 30867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:38.681730 30867 hybrid_clock.cc:648] HybridClock initialized: now 1786006658681663 us; error 60 us; skew 500 ppm
I20260806 08:57:38.682121 30867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:57:38.684610 30867 webserver.cc:460] Webserver started at http://127.22.98.62:35673/ using document root <none> and password file <none>
I20260806 08:57:38.685344 30867 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:38.685457 30867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:38.685766 30867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:38.688786 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "3e46927277764294a1ec0e5888a59b7f"
format_stamp: "Formatted at 2026-08-06 08:57:38 on dist-test-slave-jpj1"
server_key: "174a65ae7b562b71eddbe850f7c228d3"
server_key_iv: "0224e22d79d01abe8a9c6ab78ef021a0"
server_key_version: "encryptionkey@0"
I20260806 08:57:38.689616 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "3e46927277764294a1ec0e5888a59b7f"
format_stamp: "Formatted at 2026-08-06 08:57:38 on dist-test-slave-jpj1"
server_key: "174a65ae7b562b71eddbe850f7c228d3"
server_key_iv: "0224e22d79d01abe8a9c6ab78ef021a0"
server_key_version: "encryptionkey@0"
I20260806 08:57:38.694953 30867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:38.699007 30887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:38.700690 30867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:38.700901 30867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "3e46927277764294a1ec0e5888a59b7f"
format_stamp: "Formatted at 2026-08-06 08:57:38 on dist-test-slave-jpj1"
server_key: "174a65ae7b562b71eddbe850f7c228d3"
server_key_iv: "0224e22d79d01abe8a9c6ab78ef021a0"
server_key_version: "encryptionkey@0"
I20260806 08:57:38.701139 30867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:38.723315 30867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:38.724261 30867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:38.724555 30867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:38.742530 30867 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:41417
I20260806 08:57:38.742549 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:41417 every 8 connection(s)
I20260806 08:57:38.743958 30867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:38.747942 30964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:38.752527 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 30867
I20260806 08:57:38.752723 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:38.753420 22920 external_mini_cluster.cc:1428] Setting key 3d604f84517c015bc7f1c27adde802f9
I20260806 08:57:38.759510 30964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f: Bootstrap starting.
I20260806 08:57:38.762863 30964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:38.764161 30964 log.cc:824] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:38.767812 30964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f: No bootstrap required, opened a new log
I20260806 08:57:38.767884 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:38.755414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.22.98.62:41417)
0806 08:57:38.756568 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:38.756629 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:38.756680 (+ 51us) server_negotiation.cc:407] Connection header received
0806 08:57:38.758454 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:38.758479 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:38.758983 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:38.759405 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:38.760058 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.761446 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.762769 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.763627 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.763818 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:38.765084 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:38.765093 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:38.765109 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:57:38.765735 (+ 626us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:38.765838 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:57:38.765865 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:38.765986 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:38.766210 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 08:57:38.766545 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":416,"threads_started":1}
I20260806 08:57:38.772024 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 08:57:38.776577 30964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } }
I20260806 08:57:38.776949 30964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:38.777050 30964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e46927277764294a1ec0e5888a59b7f, State: Initialized, Role: FOLLOWER
I20260806 08:57:38.777598 30964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } }
I20260806 08:57:38.777809 30964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:38.777930 30964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:38.778090 30964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:38.780406 30964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } }
I20260806 08:57:38.780972 30964 leader_election.cc:302] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e46927277764294a1ec0e5888a59b7f; no voters:
I20260806 08:57:38.781711 30964 leader_election.cc:288] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:38.781840 30969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:38.782608 30969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [term 1 LEADER]: Becoming Leader. State: Replica: 3e46927277764294a1ec0e5888a59b7f, State: Running, Role: LEADER
I20260806 08:57:38.783144 30969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } }
I20260806 08:57:38.783946 30964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:38.786247 30970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e46927277764294a1ec0e5888a59b7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } } }
I20260806 08:57:38.786675 30970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:38.787503 30971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e46927277764294a1ec0e5888a59b7f. Latest consensus state: current_term: 1 leader_uuid: "3e46927277764294a1ec0e5888a59b7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e46927277764294a1ec0e5888a59b7f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41417 } } }
I20260806 08:57:38.787876 30971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:38.792587 30983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:38.798079 30983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:38.808293 30983 catalog_manager.cc:1347] Generated new cluster ID: f3a8729c94fe470ea331ad042c7f1ccb
I20260806 08:57:38.808410 30983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:38.825012 30983 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:38.826357 30983 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:38.836823 30983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e46927277764294a1ec0e5888a59b7f: Generated new TSK 0
I20260806 08:57:38.837690 30983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:57:38.962129 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:38.851744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.22.98.62:41417)
0806 08:57:38.851926 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:38.851934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:38.852350 (+ 416us) server_negotiation.cc:407] Connection header received
0806 08:57:38.852643 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:38.852648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:38.852728 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:38.852882 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0806 08:57:38.853353 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.854108 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.855550 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.856026 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.856137 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:38.856652 (+ 515us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 08:57:38.961273 (+104621us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 08:57:38.961779 (+ 506us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49624: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 08:57:38.962298 30967 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49624: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 08:57:38.962368 31000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:38.851640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41417 (local address 127.0.0.1:49624)
0806 08:57:38.852240 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:38.852257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:38.852406 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:38.852920 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:38.852925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:38.852938 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 08:57:38.853199 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:38.853209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.854266 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.854270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:38.855386 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:38.855399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:38.856209 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:38.856216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:38.856347 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:38.856381 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 08:57:38.961596 (+105215us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 08:57:38.961917 (+ 321us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.98.62:41417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":329,"threads_started":1}
I20260806 08:57:38.963989 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 30867
I20260806 08:57:38.965353 30955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:39.068581 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 30867
2026-08-06T08:57:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (852 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (434 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (436 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:40 dist-test-slave-jpj1 krb5kdc[31022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:40 dist-test-slave-jpj1 krb5kdc[31022](info): set up 2 sockets
Aug 06 08:57:40 dist-test-slave-jpj1 krb5kdc[31022](info): commencing operation
krb5kdc: starting...
W20260806 08:57:41.126205 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:41 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:57:41.384675 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42973
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:42973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:41.736997 31045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:41.737320 31045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:41.737437 31045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:41.744865 31045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:41.744992 31045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:41.745051 31045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:41.745136 31045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:41.745200 31045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:41.755579 31045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:42973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:41.756989 31045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:41.758330 31045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:41.770457 31053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:41.770583 31052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:41.770794 31056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:41.771618 31045 server_base.cc:1034] running on GCE node
I20260806 08:57:41.772508 31045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:41.774068 31045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:41.775323 31045 hybrid_clock.cc:648] HybridClock initialized: now 1786006661775256 us; error 56 us; skew 500 ppm
I20260806 08:57:41.775743 31045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:41 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:41.779021 31045 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:57:41.780596 31045 webserver.cc:460] Webserver started at http://127.22.98.62:37113/ using document root <none> and password file <none>
I20260806 08:57:41.781334 31045 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:41.781461 31045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:41.781795 31045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:41.784981 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92"
format_stamp: "Formatted at 2026-08-06 08:57:41 on dist-test-slave-jpj1"
server_key: "44da7f1934ca0cc306ac7330448eb4ea"
server_key_iv: "8edfdcdd88a54b5df187c22e4ec0e8ba"
server_key_version: "encryptionkey@0"
I20260806 08:57:41.785780 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92"
format_stamp: "Formatted at 2026-08-06 08:57:41 on dist-test-slave-jpj1"
server_key: "44da7f1934ca0cc306ac7330448eb4ea"
server_key_iv: "8edfdcdd88a54b5df187c22e4ec0e8ba"
server_key_version: "encryptionkey@0"
I20260806 08:57:41.791265 31045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 08:57:41.794943 31064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:41.796396 31045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:41.796572 31045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92"
format_stamp: "Formatted at 2026-08-06 08:57:41 on dist-test-slave-jpj1"
server_key: "44da7f1934ca0cc306ac7330448eb4ea"
server_key_iv: "8edfdcdd88a54b5df187c22e4ec0e8ba"
server_key_version: "encryptionkey@0"
I20260806 08:57:41.796775 31045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:41.808430 31045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:41.812659 31045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:41.812947 31045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:41.831091 31045 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:42973
I20260806 08:57:41.831125 31129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:42973 every 8 connection(s)
I20260806 08:57:41.832547 31045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:41.836491 31132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:41.839800 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31045
I20260806 08:57:41.840029 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:41.840883 22920 external_mini_cluster.cc:1428] Setting key 6ef055331ee026e92c86591a6ea49ec0
I20260806 08:57:41.850865 31132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92: Bootstrap starting.
I20260806 08:57:41.854718 31132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:41.855844 31132 log.cc:824] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92: Log is configured to *not* fsync() on all Append() calls
Aug 06 08:57:41 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:41.859272 31132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92: No bootstrap required, opened a new log
I20260806 08:57:41.866160 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:41.843207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.22.98.62:42973)
0806 08:57:41.844100 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:41.844115 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:41.844563 (+ 448us) server_negotiation.cc:407] Connection header received
0806 08:57:41.846541 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:41.846573 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:41.847124 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:41.847631 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:41.849004 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:41.850996 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:41.852353 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:41.853436 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:41.853633 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:41.858146 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:41.858155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:41.858171 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:41.858210 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:41.861581 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:41.862483 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:41.862492 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:41.862499 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:41.862701 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:41.863169 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:41.863174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:41.863176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:41.863892 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:41.864153 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:41.864607 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 08:57:41.865033 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":375,"threads_started":1}
I20260806 08:57:41.868487 31132 raft_consensus.cc:348] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } }
I20260806 08:57:41.868870 31132 raft_consensus.cc:374] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:41.868978 31132 raft_consensus.cc:729] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afca8c9f0a194e3697bbb8ca2cbb7a92, State: Initialized, Role: FOLLOWER
I20260806 08:57:41.869614 31132 consensus_queue.cc:260] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } }
I20260806 08:57:41.869867 31132 raft_consensus.cc:388] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:41.870007 31132 raft_consensus.cc:482] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:41.870172 31132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:41.872490 31132 raft_consensus.cc:504] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } }
I20260806 08:57:41.873019 31132 leader_election.cc:302] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afca8c9f0a194e3697bbb8ca2cbb7a92; no voters:
I20260806 08:57:41.873559 31132 leader_election.cc:288] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:41.873690 31138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:41.874445 31138 raft_consensus.cc:686] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [term 1 LEADER]: Becoming Leader. State: Replica: afca8c9f0a194e3697bbb8ca2cbb7a92, State: Running, Role: LEADER
I20260806 08:57:41.874984 31138 consensus_queue.cc:237] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } }
I20260806 08:57:41.875913 31132 sys_catalog.cc:564] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:41.877960 31139 sys_catalog.cc:455] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } } }
I20260806 08:57:41.877935 31143 sys_catalog.cc:455] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afca8c9f0a194e3697bbb8ca2cbb7a92. Latest consensus state: current_term: 1 leader_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afca8c9f0a194e3697bbb8ca2cbb7a92" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42973 } } }
I20260806 08:57:41.878439 31139 sys_catalog.cc:458] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:41.878494 31143 sys_catalog.cc:458] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:41.880613 31147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:41.885210 31147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:41.895272 31147 catalog_manager.cc:1347] Generated new cluster ID: f8a5e685224a4ca18c3c37fcffef75f3
I20260806 08:57:41.895403 31147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:41.919643 31147 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:41.921056 31147 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:41.932325 31147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afca8c9f0a194e3697bbb8ca2cbb7a92: Generated new TSK 0
I20260806 08:57:41.933226 31147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:57:42.010980 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:42.403347 31170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:42.403723 31170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:42.403872 31170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:42.411484 31170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:42.411684 31170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:42.411857 31170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:42.422586 31170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:42.424925 31170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:42.426518 31170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:42.441174 31179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:42.441210 31178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:42.441149 31183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:42.442133 31170 server_base.cc:1034] running on GCE node
I20260806 08:57:42.442967 31170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:42.445057 31170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:42.446316 31170 hybrid_clock.cc:648] HybridClock initialized: now 1786006662446247 us; error 52 us; skew 500 ppm
I20260806 08:57:42.446853 31170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:42 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006662, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:42.453996 31170 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:57:42.455902 31170 webserver.cc:460] Webserver started at http://127.22.98.1:41219/ using document root <none> and password file <none>
I20260806 08:57:42.456691 31170 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:42.456833 31170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:42.457165 31170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:42.460578 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "b1b551f53fa44c86b95f348693b68217"
format_stamp: "Formatted at 2026-08-06 08:57:42 on dist-test-slave-jpj1"
server_key: "dbd84728d9a843940a516dbc1b633c69"
server_key_iv: "83406a90109b579b277ee2ee41d9642a"
server_key_version: "encryptionkey@0"
I20260806 08:57:42.461503 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "b1b551f53fa44c86b95f348693b68217"
format_stamp: "Formatted at 2026-08-06 08:57:42 on dist-test-slave-jpj1"
server_key: "dbd84728d9a843940a516dbc1b633c69"
server_key_iv: "83406a90109b579b277ee2ee41d9642a"
server_key_version: "encryptionkey@0"
I20260806 08:57:42.468292 31170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 08:57:42.472597 31191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:42.474188 31170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 08:57:42.474382 31170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "b1b551f53fa44c86b95f348693b68217"
format_stamp: "Formatted at 2026-08-06 08:57:42 on dist-test-slave-jpj1"
server_key: "dbd84728d9a843940a516dbc1b633c69"
server_key_iv: "83406a90109b579b277ee2ee41d9642a"
server_key_version: "encryptionkey@0"
I20260806 08:57:42.474609 31170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:42.512173 31170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:42.516615 31170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:42.516938 31170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:42.517961 31170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:42.519426 31170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:42.519516 31170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:42.519670 31170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:42.519727 31170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:42.551637 31170 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:41651
I20260806 08:57:42.551683 31347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:41651 every 8 connection(s)
I20260806 08:57:42.553020 31170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:42.562269 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31170
I20260806 08:57:42.562484 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:42.563339 22920 external_mini_cluster.cc:1428] Setting key f1f26d02f38269be207b479631491643
Aug 06 08:57:42 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006662, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:42.574394 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:42.555684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:43015 (local address 127.22.98.62:42973)
0806 08:57:42.556163 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:42.556170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:42.557196 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 08:57:42.559049 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:42.559054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:42.559127 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:42.559233 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:42.560810 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:42.561581 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:42.563973 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:42.564527 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:42.564670 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:42.568499 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:42.568506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:42.568509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:42.568545 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:42.571089 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:42.571969 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:42.571974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:42.571976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:42.572109 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:42.572561 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:42.572565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:42.572567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:42.572858 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:42.572998 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:42.573787 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 08:57:42.574037 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":216,"threads_started":1}
I20260806 08:57:42.575444 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:42.556229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42973 (local address 127.22.98.1:43015)
0806 08:57:42.557025 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:42.557070 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:42.558749 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:42.559429 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:42.559438 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:42.560042 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:42.560628 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:42.560646 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:42.561741 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:42.561745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:42.563792 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:42.563810 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:42.564782 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:42.564790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:42.565056 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:42.565817 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:42.565842 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:42.568247 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:42.571241 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:42.571252 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:42.571272 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:42.571835 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:42.572226 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:42.572231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:42.572235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:42.572438 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:42.572992 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:42.573001 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:42.573379 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 08:57:42.573630 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 08:57:42.574216 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":355,"threads_started":1}
I20260806 08:57:42.577221 31348 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42973
I20260806 08:57:42.577566 31348 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:42.578395 31348 heartbeater.cc:511] Master 127.22.98.62:42973 requested a full tablet report, sending...
I20260806 08:57:42.580911 31086 ts_manager.cc:194] Registered new tserver with Master: b1b551f53fa44c86b95f348693b68217 (127.22.98.1:41651)
I20260806 08:57:42.582969 31086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:43015
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:57:42.619665 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:42.964180 31356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:42.964490 31356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:42.964587 31356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:42.972024 31356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:42.972162 31356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:42.972340 31356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:42.982779 31356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:42.984241 31356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:42.985603 31356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:42.998857 31361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:42.998895 31362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:42.999173 31367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:43.000075 31356 server_base.cc:1034] running on GCE node
I20260806 08:57:43.000787 31356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:43.001823 31356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:43.003055 31356 hybrid_clock.cc:648] HybridClock initialized: now 1786006663002998 us; error 39 us; skew 500 ppm
I20260806 08:57:43.003391 31356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.007004 31356 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:57:43.008713 31356 webserver.cc:460] Webserver started at http://127.22.98.2:34709/ using document root <none> and password file <none>
I20260806 08:57:43.009475 31356 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:43.009618 31356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:43.009949 31356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:43.013175 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "cbc2ba08ca994c409c71f75412752379"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "c583e8f56745dbb2399030f5b220a33f"
server_key_iv: "8490ba6f3fc34603077d27e98ca8d0a1"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.013996 31356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "cbc2ba08ca994c409c71f75412752379"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "c583e8f56745dbb2399030f5b220a33f"
server_key_iv: "8490ba6f3fc34603077d27e98ca8d0a1"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.020008 31356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 08:57:43.023947 31376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.025925 31356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 08:57:43.026134 31356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "cbc2ba08ca994c409c71f75412752379"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "c583e8f56745dbb2399030f5b220a33f"
server_key_iv: "8490ba6f3fc34603077d27e98ca8d0a1"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.026369 31356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:43.048380 31356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:43.052922 31356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:43.053304 31356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:43.054477 31356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:43.056039 31356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:43.056129 31356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.056257 31356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:43.056318 31356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.088903 31356 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46309
I20260806 08:57:43.088964 31533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46309 every 8 connection(s)
I20260806 08:57:43.090320 31356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:43.096637 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31356
I20260806 08:57:43.096851 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:43.097318 22920 external_mini_cluster.cc:1428] Setting key efa9c2df4d6ff19813ba1adf980a8915
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:43.111716 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.093014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:44601 (local address 127.22.98.62:42973)
0806 08:57:43.093477 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.093482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.094486 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 08:57:43.096471 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.096477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.096564 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.096666 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.098540 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.099462 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.101212 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.101731 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.101856 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.105519 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.105524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.105528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.105564 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.108388 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.109225 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.109230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.109232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.109384 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.109852 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.109856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.109859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.110142 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.110284 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.111040 (+ 756us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.111295 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260806 08:57:43.112833 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.093488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42973 (local address 127.22.98.2:44601)
0806 08:57:43.094299 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:43.094342 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:43.096159 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:43.096961 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:43.096974 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:43.097600 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:43.098353 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:43.098376 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.099648 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.099652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:43.101058 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:43.101072 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.101988 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.101996 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:43.102234 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.102924 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:43.102956 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:43.105248 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:43.108573 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:43.108582 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:43.108599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:43.109087 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:43.109501 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:43.109506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:43.109508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:43.109728 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:43.110277 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:43.110286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:43.110666 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 08:57:43.110929 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 08:57:43.111527 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":313,"threads_started":1}
I20260806 08:57:43.114529 31534 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42973
I20260806 08:57:43.114948 31534 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:43.115919 31534 heartbeater.cc:511] Master 127.22.98.62:42973 requested a full tablet report, sending...
I20260806 08:57:43.117817 31086 ts_manager.cc:194] Registered new tserver with Master: cbc2ba08ca994c409c71f75412752379 (127.22.98.2:46309)
I20260806 08:57:43.118994 31086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:44601
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:57:43.155436 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:43.502857 31540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:43.503165 31540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:43.503263 31540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:43.510708 31540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:43.510839 31540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:43.510957 31540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:43.521170 31540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:43.522478 31540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:43.523962 31540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:43.537197 31546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:43.537305 31547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:43.537230 31550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:43.538367 31540 server_base.cc:1034] running on GCE node
I20260806 08:57:43.539077 31540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:43.540138 31540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:43.541356 31540 hybrid_clock.cc:648] HybridClock initialized: now 1786006663541285 us; error 62 us; skew 500 ppm
I20260806 08:57:43.541685 31540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.545161 31540 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:57:43.546931 31540 webserver.cc:460] Webserver started at http://127.22.98.3:44413/ using document root <none> and password file <none>
I20260806 08:57:43.547777 31540 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:43.547919 31540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:43.548260 31540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:43.551783 31540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "3f2a5c034d06433a98790fff2c2851cc"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "55563d18c32b68a236bfc811001722e3"
server_key_iv: "368e32e6e42ce8118697ce81ab3febfc"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.552874 31540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "3f2a5c034d06433a98790fff2c2851cc"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "55563d18c32b68a236bfc811001722e3"
server_key_iv: "368e32e6e42ce8118697ce81ab3febfc"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.559712 31540 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 08:57:43.563403 31559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.564917 31540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:43.565088 31540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "3f2a5c034d06433a98790fff2c2851cc"
format_stamp: "Formatted at 2026-08-06 08:57:43 on dist-test-slave-jpj1"
server_key: "55563d18c32b68a236bfc811001722e3"
server_key_iv: "368e32e6e42ce8118697ce81ab3febfc"
server_key_version: "encryptionkey@0"
I20260806 08:57:43.565300 31540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:43.584347 31540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:43.586822 31348 heartbeater.cc:503] Master 127.22.98.62:42973 was elected leader, sending a full tablet report...
I20260806 08:57:43.589252 31540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:43.589551 31540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:43.590674 31540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:43.592216 31540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:43.592306 31540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.592427 31540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:43.592473 31540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:43.624689 31540 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:43441
I20260806 08:57:43.624765 31713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:43441 every 8 connection(s)
I20260806 08:57:43.626019 31540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:43.631494 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31540
I20260806 08:57:43.631688 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:43.632140 22920 external_mini_cluster.cc:1428] Setting key 7f7c1732e90142881c95e23b2a3d08c9
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:57:43.648237 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.628629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59443 (local address 127.22.98.62:42973)
0806 08:57:43.629125 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.629130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.630016 (+ 886us) server_negotiation.cc:407] Connection header received
0806 08:57:43.632023 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.632029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.632122 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.632217 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.633963 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.634872 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.637203 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.637820 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.637956 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.642170 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.642176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.642179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.642216 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.644685 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.645459 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.645465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.645467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.645596 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.646129 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.646135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.646138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.646510 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.646722 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.647588 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.647849 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":273,"threads_started":1}
I20260806 08:57:43.649224 31719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.629043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42973 (local address 127.22.98.3:59443)
0806 08:57:43.629841 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:43.629904 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:43.631718 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:43.632484 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:43.632493 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:43.633146 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:43.633797 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:43.633817 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.635052 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.635057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:43.636994 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:43.637010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.638716 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.638729 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:43.639067 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.639741 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:43.639771 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:43.641935 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:43.644843 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:43.644853 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:43.644872 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:43.645323 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:43.645707 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:43.645712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:43.645716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:43.645971 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:43.646657 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:43.646665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:43.647086 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 08:57:43.647410 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 08:57:43.648025 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":299,"threads_started":1}
I20260806 08:57:43.650741 31714 heartbeater.cc:347] Connected to a master server at 127.22.98.62:42973
I20260806 08:57:43.651046 31714 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:43.651830 31714 heartbeater.cc:511] Master 127.22.98.62:42973 requested a full tablet report, sending...
I20260806 08:57:43.653364 31086 ts_manager.cc:194] Registered new tserver with Master: 3f2a5c034d06433a98790fff2c2851cc (127.22.98.3:43441)
I20260806 08:57:43.654214 31086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:59443
I20260806 08:57:43.661007 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/test-admin.keytab.
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.693562 22920 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 08:57:43.699211 31725 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:43.696485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.22.98.1:41651)
0806 08:57:43.697270 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.697277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.697304 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:57:43.697377 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.697384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.697596 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.697763 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.698791 (+ 1028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56446: BlockingRecv error: recv got EOF from 127.0.0.1:56446 (error 108)
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":415,"threads_started":1}
W20260806 08:57:43.699370 22920 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc' of type FILE
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.701184 22920 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.702724 22920 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:43.718780 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.704458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.22.98.1:41651)
0806 08:57:43.704688 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.704694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.704970 (+ 276us) server_negotiation.cc:407] Connection header received
0806 08:57:43.705145 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.705149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.705217 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.705310 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.706409 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.707469 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.708850 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.709436 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.709535 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.713114 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.713127 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.713145 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.713197 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.715955 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.716646 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.716655 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.716657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.716851 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.717166 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.717169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.717171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.717545 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.717799 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.718124 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.718480 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 08:57:43.725003 31725 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:43.723122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.22.98.1:41651)
0806 08:57:43.723349 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.723354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.723523 (+ 169us) server_negotiation.cc:407] Connection header received
0806 08:57:43.723699 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.723702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.723772 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.723857 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.724737 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56464: BlockingRecv error: recv got EOF from 127.0.0.1:56464 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 08:57:43.725126 22920 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc' of type FILE
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.726675 22920 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.727996 22920 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:43.742775 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.729325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.22.98.1:41651)
0806 08:57:43.729522 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.729528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.729837 (+ 309us) server_negotiation.cc:407] Connection header received
0806 08:57:43.729962 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.729966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.730028 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.730127 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.731342 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.732136 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.733373 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.733812 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.733916 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.737611 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.737617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.737621 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.737676 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.740290 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.740989 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.740993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.740996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.741126 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.741469 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.741474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.741477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.741783 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.742002 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.742277 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.742457 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 08:57:43.747014 31725 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:43.745439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.22.98.1:41651)
0806 08:57:43.745637 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.745643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.745860 (+ 217us) server_negotiation.cc:407] Connection header received
0806 08:57:43.745994 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.745998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.746077 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.746147 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.746812 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56482: BlockingRecv error: recv got EOF from 127.0.0.1:56482 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 08:57:43.747154 22920 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786006610327111-22920-0/krb5kdc/krb5cc' of type FILE
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.748775 22920 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:43.750094 22920 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:57:43.764842 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.751719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.22.98.1:41651)
0806 08:57:43.751932 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.751938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.752019 (+ 81us) server_negotiation.cc:407] Connection header received
0806 08:57:43.752235 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.752239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.752305 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.752389 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.753403 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.754136 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.755372 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.755835 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.755949 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.759617 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.759622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.759626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.759662 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.762124 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.762842 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.762846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.762849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.763010 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.763377 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.763381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.763383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.763723 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.764022 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.764289 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.764512 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 08:57:43.766717 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31170
I20260806 08:57:43.779785 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.767459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.22.98.1:41651)
0806 08:57:43.767695 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.767701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.767992 (+ 291us) server_negotiation.cc:407] Connection header received
0806 08:57:43.768126 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.768130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.768186 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.768278 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.769298 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.770117 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.771394 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.771926 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.772024 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.774144 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.774150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.774153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.774187 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.776815 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.777538 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.777542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.777545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.777701 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.778093 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.778098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.778100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.778479 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.778690 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.779104 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.779328 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 08:57:43.780447 31341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:43.864444 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31170
I20260806 08:57:43.885627 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31356
Aug 06 08:57:43 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:57:43.902014 31743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:43.886485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.22.98.2:46309)
0806 08:57:43.886993 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:43.887001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:43.887028 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:57:43.887091 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:43.887097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:43.887317 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:43.887472 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:43.888680 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.889816 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.891232 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:43.892038 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:43.892152 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:43.896166 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:43.896181 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:43.896197 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:43.896234 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:43.899125 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.899943 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.899951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.899954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.900094 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:43.900477 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:43.900480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:43.900482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:43.900891 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:43.901145 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:43.901458 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 08:57:43.901682 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":249,"threads_started":1}
I20260806 08:57:43.903385 31529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:43.985270 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31356
I20260806 08:57:44.006243 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31540
Aug 06 08:57:44 dist-test-slave-jpj1 krb5kdc[31022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:57:44.023247 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:44.007308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.22.98.3:43441)
0806 08:57:44.008133 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:44.008141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:44.008174 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:57:44.008269 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:44.008276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:44.008500 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:44.008678 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:44.009867 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:44.010947 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:44.012254 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:44.012851 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:44.012972 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:44.017321 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:44.017334 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:44.017352 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:44.017407 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:44.020319 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:44.021083 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:44.021091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:44.021093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:44.021229 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:44.021617 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:44.021621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:44.021623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:44.021997 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:44.022326 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:44.022651 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 08:57:44.022920 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":489,"threads_started":1}
I20260806 08:57:44.024694 31708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:44.107376 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31540
I20260806 08:57:44.128038 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31045
I20260806 08:57:44.129359 31123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:44.233038 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31045
2026-08-06T08:57:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4294 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:44 dist-test-slave-jpj1 krb5kdc[31756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:44 dist-test-slave-jpj1 krb5kdc[31756](info): set up 2 sockets
Aug 06 08:57:44 dist-test-slave-jpj1 krb5kdc[31756](info): commencing operation
krb5kdc: starting...
W20260806 08:57:45.413774 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:45 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab.
Entry for principal oryx/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab.
I20260806 08:57:45.657387 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40807
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab
--principal=oryx/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:45.995383 31775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:45.995764 31775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:45.995909 31775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:46.003165 31775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:46.003304 31775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:46.003376 31775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:46.003455 31775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:46.003518 31775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:46.013731 31775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab
--principal=oryx/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:46.015204 31775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:46.016638 31775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:46.028714 31780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:46.028764 31784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:46.028805 31781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:46.030104 31775 server_base.cc:1034] running on GCE node
I20260806 08:57:46.030901 31775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:46.032431 31775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:46.033641 31775 hybrid_clock.cc:648] HybridClock initialized: now 1786006666033596 us; error 34 us; skew 500 ppm
I20260806 08:57:46.033993 31775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:46 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006666, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:46.037140 31775 init.cc:377] Logged in from keytab as oryx/127.22.98.62@KRBTEST.COM (short username oryx)
I20260806 08:57:46.038663 31775 webserver.cc:460] Webserver started at http://127.22.98.62:38241/ using document root <none> and password file <none>
I20260806 08:57:46.039345 31775 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:46.039451 31775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:46.039772 31775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:46.042795 31775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "b7e1a719411b4d1f907a0147a3870384"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "54f91881a68e1f68551f033f957a0c36"
server_key_iv: "8c7e2a7599ab55dee4165ab56a530247"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.043515 31775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "b7e1a719411b4d1f907a0147a3870384"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "54f91881a68e1f68551f033f957a0c36"
server_key_iv: "8c7e2a7599ab55dee4165ab56a530247"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.048828 31775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:57:46.052551 31793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:46.054095 31775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 08:57:46.054275 31775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "b7e1a719411b4d1f907a0147a3870384"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "54f91881a68e1f68551f033f957a0c36"
server_key_iv: "8c7e2a7599ab55dee4165ab56a530247"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.054483 31775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:46.079478 31775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:46.084553 31775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:46.084908 31775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:46.104167 31775 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:40807
I20260806 08:57:46.104188 31860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:40807 every 8 connection(s)
I20260806 08:57:46.105587 31775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:46.109474 31861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:46.112427 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31775
I20260806 08:57:46.112635 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:46.113284 22920 external_mini_cluster.cc:1428] Setting key 7ed332ab8ca435427f352915bf50261c
I20260806 08:57:46.122078 31861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Bootstrap starting.
I20260806 08:57:46.125989 31861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:46.127341 31861 log.cc:824] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Log is configured to *not* fsync() on all Append() calls
Aug 06 08:57:46 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:46.130597 31861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: No bootstrap required, opened a new log
I20260806 08:57:46.136983 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:46.115384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.22.98.62:40807)
0806 08:57:46.116219 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:46.116232 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:46.116277 (+ 45us) server_negotiation.cc:407] Connection header received
0806 08:57:46.117959 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:46.117986 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:46.118435 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:46.118803 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:46.120642 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.121982 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.123747 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.124464 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.124594 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:46.129459 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:46.129469 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:46.129485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:46.129527 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:46.132675 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:46.133385 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:46.133393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:46.133400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:46.133581 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:46.133989 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:46.133993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:46.133996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:46.134581 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:46.134939 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:46.135384 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 08:57:46.135886 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":401,"threads_started":1}
I20260806 08:57:46.139042 31861 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } }
I20260806 08:57:46.139415 31861 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:46.139509 31861 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e1a719411b4d1f907a0147a3870384, State: Initialized, Role: FOLLOWER
I20260806 08:57:46.140136 31861 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } }
I20260806 08:57:46.140422 31861 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:46.140554 31861 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:46.140712 31861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:46.143134 31861 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } }
I20260806 08:57:46.143719 31861 leader_election.cc:302] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e1a719411b4d1f907a0147a3870384; no voters:
I20260806 08:57:46.144312 31861 leader_election.cc:288] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:46.144454 31867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:46.145123 31867 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 LEADER]: Becoming Leader. State: Replica: b7e1a719411b4d1f907a0147a3870384, State: Running, Role: LEADER
I20260806 08:57:46.145725 31867 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } }
I20260806 08:57:46.146487 31861 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:46.148834 31868 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } }
I20260806 08:57:46.148869 31869 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e1a719411b4d1f907a0147a3870384. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } }
I20260806 08:57:46.149253 31869 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:46.149251 31868 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:46.150640 31874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:46.156370 31874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:46.166416 31874 catalog_manager.cc:1347] Generated new cluster ID: 0d9ac1548a2d4b29a063d8f9ab8eadec
I20260806 08:57:46.166544 31874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:46.181988 31874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:46.183349 31874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:46.192682 31874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Generated new TSK 0
I20260806 08:57:46.193629 31874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
I20260806 08:57:46.256068 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:46.613601 31896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:46.613911 31896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:46.614037 31896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:46.621362 31896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:46.621486 31896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:46.621613 31896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:46.631848 31896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:46.633343 31896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:46.634855 31896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:46.648890 31903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:46.648837 31904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:46.649477 31906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:46.650995 31896 server_base.cc:1034] running on GCE node
I20260806 08:57:46.651731 31896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:46.652774 31896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:46.653986 31896 hybrid_clock.cc:648] HybridClock initialized: now 1786006666653950 us; error 48 us; skew 500 ppm
I20260806 08:57:46.654311 31896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:46 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006666, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:46.657435 31896 init.cc:377] Logged in from keytab as oryx/127.22.98.1@KRBTEST.COM (short username oryx)
I20260806 08:57:46.658882 31896 webserver.cc:460] Webserver started at http://127.22.98.1:44409/ using document root <none> and password file <none>
I20260806 08:57:46.659685 31896 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:46.659795 31896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:46.660071 31896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:46.662993 31896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "4f72a334b34340958f1ba0d3171f79a6"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "9b17cede0e5400029d9902b54a3a9963"
server_key_iv: "8a0e972cd677393ec5cbc378120c8c33"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.663781 31896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "4f72a334b34340958f1ba0d3171f79a6"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "9b17cede0e5400029d9902b54a3a9963"
server_key_iv: "8a0e972cd677393ec5cbc378120c8c33"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.669104 31896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:46.672677 31914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:46.674283 31896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:46.674436 31896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "4f72a334b34340958f1ba0d3171f79a6"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "9b17cede0e5400029d9902b54a3a9963"
server_key_iv: "8a0e972cd677393ec5cbc378120c8c33"
server_key_version: "encryptionkey@0"
I20260806 08:57:46.674623 31896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:46.690960 31896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:46.695585 31896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:46.695873 31896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:46.696862 31896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:46.698338 31896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:46.698413 31896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:46.698508 31896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:46.698556 31896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:46.730746 31896 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:32931
I20260806 08:57:46.730782 32056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:32931 every 8 connection(s)
I20260806 08:57:46.732103 31896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:46.736550 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 31896
I20260806 08:57:46.736760 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:46.737396 22920 external_mini_cluster.cc:1428] Setting key b13de4f4247e2a28b7b3289f6010b349
Aug 06 08:57:46 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006666, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:46.751890 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:46.734715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:55293 (local address 127.22.98.62:40807)
0806 08:57:46.735152 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:46.735157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:46.736189 (+ 1032us) server_negotiation.cc:407] Connection header received
0806 08:57:46.738094 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:46.738100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:46.738187 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:46.738298 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:46.739809 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.740515 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.742058 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.742446 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.742559 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:46.745955 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:46.745960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:46.745964 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:46.745997 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:46.748520 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:46.749326 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:46.749331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:46.749334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:46.749466 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:46.749986 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:46.749991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:46.749993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:46.750295 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:46.750410 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:46.751356 (+ 946us) server_negotiation.cc:299] Negotiation successful
0806 08:57:46.751610 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":214,"threads_started":1}
I20260806 08:57:46.753245 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:46.735283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:55293)
0806 08:57:46.736036 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:46.736084 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:46.737826 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:46.738537 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:46.738545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:46.739049 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:46.739651 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:46.739670 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.740663 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.740667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:46.741914 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:46.741927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:46.742676 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:46.742683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:46.742922 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:46.743585 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:46.743611 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:46.745728 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:46.748652 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:46.748662 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:46.748680 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:46.749188 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:46.749581 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:46.749587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:46.749591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:46.749848 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:46.750420 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:46.750428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:46.750870 (+ 442us) client_negotiation.cc:769] Sending connection context
0806 08:57:46.751231 (+ 361us) client_negotiation.cc:240] Negotiation successful
0806 08:57:46.751808 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":298,"threads_started":1}
I20260806 08:57:46.754900 32058 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:46.755240 32058 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:46.756250 32058 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:46.758602 31814 ts_manager.cc:194] Registered new tserver with Master: 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:46.760226 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:55293
WARNING: no policy specified for oryx/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
I20260806 08:57:46.794816 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:47.143069 32070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:47.143384 32070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:47.143514 32070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:47.151275 32070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:47.151402 32070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:47.151571 32070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:47.161998 32070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:47.163399 32070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:47.164860 32070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:47.179117 32080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:47.179141 32082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:47.179049 32079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:47.180246 32070 server_base.cc:1034] running on GCE node
I20260806 08:57:47.180930 32070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:47.182032 32070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:47.183250 32070 hybrid_clock.cc:648] HybridClock initialized: now 1786006667183186 us; error 55 us; skew 500 ppm
I20260806 08:57:47.183594 32070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:47 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006667, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:47.187002 32070 init.cc:377] Logged in from keytab as oryx/127.22.98.2@KRBTEST.COM (short username oryx)
I20260806 08:57:47.188583 32070 webserver.cc:460] Webserver started at http://127.22.98.2:42667/ using document root <none> and password file <none>
I20260806 08:57:47.189383 32070 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:47.189510 32070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:47.189849 32070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:47.193285 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "675f008bf969f4c62ba0ce7b301fc3b0"
server_key_iv: "fd41fb54e228e5a0c537297940786159"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.194204 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "675f008bf969f4c62ba0ce7b301fc3b0"
server_key_iv: "fd41fb54e228e5a0c537297940786159"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.200594 32070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 08:57:47.204977 32091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.206621 32070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260806 08:57:47.206844 32070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "675f008bf969f4c62ba0ce7b301fc3b0"
server_key_iv: "fd41fb54e228e5a0c537297940786159"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.207100 32070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:47.224488 32070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:47.229161 32070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:47.229471 32070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:47.230723 32070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:47.232422 32070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:47.232515 32070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.232645 32070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:47.232704 32070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.266295 32070 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39299
I20260806 08:57:47.266340 32239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39299 every 8 connection(s)
I20260806 08:57:47.267663 32070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:47.270977 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 32070
I20260806 08:57:47.271147 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:47.271641 22920 external_mini_cluster.cc:1428] Setting key 4d752aa1d343deec018ae4511a35e99a
Aug 06 08:57:47 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006667, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:47.289785 32243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:47.270332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:59819 (local address 127.22.98.62:40807)
0806 08:57:47.270840 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:47.270847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:47.272004 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 08:57:47.274099 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:47.274105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:47.274200 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:47.274332 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:47.276086 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.277055 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.278851 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.279361 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.279498 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:47.283402 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:47.283408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:47.283411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:47.283446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:47.286019 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:47.286945 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:47.286948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:47.286951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:47.287082 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:47.287761 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:47.287765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:47.287768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:47.288117 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:47.288255 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:47.289268 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0806 08:57:47.289462 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260806 08:57:47.291234 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:47.270920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:59819)
0806 08:57:47.271805 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:47.271854 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:47.273816 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:47.274606 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:47.274617 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:47.275189 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:47.275894 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:47.275916 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.277236 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.277241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:47.278649 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:47.278662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.279624 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.279632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:47.279901 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:47.280555 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:47.280579 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:47.283127 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:47.286172 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:47.286181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:47.286232 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:47.286829 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:47.287185 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:47.287196 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:47.287199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:47.287515 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:47.288269 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:47.288277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:47.288750 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 08:57:47.289135 (+ 385us) client_negotiation.cc:240] Negotiation successful
0806 08:57:47.289731 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":343,"threads_started":1}
I20260806 08:57:47.293074 32240 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:47.293426 32240 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:47.294309 32240 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:47.295964 31814 ts_manager.cc:194] Registered new tserver with Master: 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299)
I20260806 08:57:47.296819 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:59819
WARNING: no policy specified for oryx/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
I20260806 08:57:47.330940 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:47.686842 32249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:47.687160 32249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:47.687305 32249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:47.694758 32249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:47.694903 32249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:47.695066 32249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:47.705856 32249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:47.707238 32249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:47.708693 32249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:47.722409 32260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:47.722388 32265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:47.722343 32263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:47.723205 32249 server_base.cc:1034] running on GCE node
I20260806 08:57:47.723882 32249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:47.724841 32249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:47.726048 32249 hybrid_clock.cc:648] HybridClock initialized: now 1786006667725984 us; error 51 us; skew 500 ppm
I20260806 08:57:47.726358 32249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:47 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006667, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:47.734591 32249 init.cc:377] Logged in from keytab as oryx/127.22.98.3@KRBTEST.COM (short username oryx)
I20260806 08:57:47.736245 32249 webserver.cc:460] Webserver started at http://127.22.98.3:34855/ using document root <none> and password file <none>
I20260806 08:57:47.736960 32249 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:47.737075 32249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:47.737429 32249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:47.740587 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "4cf77181c0df4751b893340a4d0f37a8"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "6ea07da81ee713ac6f50841abe7e3824"
server_key_iv: "2b3453ed0ec4bb20b1273a6054a4321e"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.741397 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "4cf77181c0df4751b893340a4d0f37a8"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "6ea07da81ee713ac6f50841abe7e3824"
server_key_iv: "2b3453ed0ec4bb20b1273a6054a4321e"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.746805 32249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:57:47.750509 32275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.752161 32249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 08:57:47.752334 32249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "4cf77181c0df4751b893340a4d0f37a8"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "6ea07da81ee713ac6f50841abe7e3824"
server_key_iv: "2b3453ed0ec4bb20b1273a6054a4321e"
server_key_version: "encryptionkey@0"
I20260806 08:57:47.752544 32249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:47.763741 32058 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
I20260806 08:57:47.771025 32249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:47.775419 32249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:47.775732 32249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:47.777034 32249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:47.778992 32249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:47.779091 32249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.779235 32249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:47.779299 32249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:47.825279 32249 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:35517
I20260806 08:57:47.825316 32427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:35517 every 8 connection(s)
I20260806 08:57:47.827184 32249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:47.828022 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 32249
I20260806 08:57:47.828214 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:47.828809 22920 external_mini_cluster.cc:1428] Setting key 448a578234cd3986457aae309454120e
Aug 06 08:57:47 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006667, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:47.850914 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:47.830449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:46757 (local address 127.22.98.62:40807)
0806 08:57:47.830961 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:47.830968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:47.834901 (+ 3933us) server_negotiation.cc:407] Connection header received
0806 08:57:47.834985 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:47.834991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:47.835085 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:47.835196 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:47.837635 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.838819 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.840544 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.840964 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.841086 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:47.845048 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:47.845054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:47.845057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:47.845094 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:47.847880 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:47.848657 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:47.848662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:47.848665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:47.848814 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:47.849235 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:47.849239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:47.849241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:47.849524 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:47.849652 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:47.850420 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 08:57:47.850628 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":541,"threads_started":1}
I20260806 08:57:47.852080 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:47.831109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:46757)
0806 08:57:47.832049 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:47.832105 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:47.834471 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:47.835583 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:47.835598 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:47.836473 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:47.837428 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:47.837462 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.839003 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.839007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:47.840378 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:47.840393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:47.841202 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:47.841210 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:47.841483 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:47.842123 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:47.842147 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:47.844739 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:47.848021 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:47.848030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:47.848047 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:47.848530 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:47.848925 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:47.848929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:47.848931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:47.849140 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:47.849636 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:47.849642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:47.849997 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 08:57:47.850232 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 08:57:47.850669 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":389,"threads_started":1}
I20260806 08:57:47.853803 32429 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:47.854144 32429 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:47.854969 32429 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:47.856652 31814 ts_manager.cc:194] Registered new tserver with Master: 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:47.857674 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:46757
I20260806 08:57:47.858310 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.98.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.61.keytab.
Entry for principal oryx/127.22.98.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.61.keytab.
I20260806 08:57:47.918624 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:36891
--webserver_interface=127.22.98.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.61.keytab
--principal=oryx/127.22.98.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:40807,127.22.98.61:36891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:48.262429 32441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:48.262746 32441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:48.262876 32441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:48.270340 32441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:48.270458 32441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:48.270537 32441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:48.270615 32441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:48.270673 32441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:48.280967 32441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:40807,127.22.98.61:36891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.61.keytab
--principal=oryx/127.22.98.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.98.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
W20260806 08:57:48.281363 32441 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.98.62:40807,127.22.98.61:36891'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 08:57:48.282414 32441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:48.283816 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:48.295454 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.295641 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.295509 32454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:48.296500 32441 server_base.cc:1034] running on GCE node
I20260806 08:57:48.297267 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:48.298611 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:48.299765 32240 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
I20260806 08:57:48.299861 32441 hybrid_clock.cc:648] HybridClock initialized: now 1786006668299802 us; error 31 us; skew 500 ppm
I20260806 08:57:48.300277 32441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:48.303768 32441 init.cc:377] Logged in from keytab as oryx/127.22.98.61@KRBTEST.COM (short username oryx)
I20260806 08:57:48.305243 32441 webserver.cc:460] Webserver started at http://127.22.98.61:43927/ using document root <none> and password file <none>
I20260806 08:57:48.305938 32441 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:48.306044 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:48.306365 32441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:48.309381 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/instance:
uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "ca7a936155573f69c6bd46762930b857"
server_key_iv: "2e27fe76ce6f4417bbc3729767ab3c8b"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.310166 32441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal/instance:
uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "ca7a936155573f69c6bd46762930b857"
server_key_iv: "2e27fe76ce6f4417bbc3729767ab3c8b"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.315755 32441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 08:57:48.319260 32466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:48.320747 32441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:48.320912 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "ca7a936155573f69c6bd46762930b857"
server_key_iv: "2e27fe76ce6f4417bbc3729767ab3c8b"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.321115 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:48.347208 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:48.351668 32441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:48.351934 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:48.369786 32441 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.61:36891
I20260806 08:57:48.369810 32533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.61:36891 every 8 connection(s)
I20260806 08:57:48.371136 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
I20260806 08:57:48.373781 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 32441
I20260806 08:57:48.375382 32534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:48.383361 32534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.61@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:48.396811 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.376047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.22.98.61:36891)
0806 08:57:48.376794 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.376807 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.376847 (+ 40us) server_negotiation.cc:407] Connection header received
0806 08:57:48.378457 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.378479 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.378915 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.379294 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.380784 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.382107 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.383310 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.384156 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.384319 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.388973 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.388984 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.388998 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.389046 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.392468 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.393265 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.393272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.393279 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.393453 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.393833 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.393836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.393838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.394364 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.394645 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.395155 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.395713 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
I20260806 08:57:48.401536 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.385963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.22.98.62:40807)
0806 08:57:48.386431 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.386437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.386947 (+ 510us) server_negotiation.cc:407] Connection header received
0806 08:57:48.387128 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.387134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.387210 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.387297 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.388775 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.389850 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.391464 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.392102 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.392238 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.395347 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.395353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.395356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.395393 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.398506 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.399382 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.399385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.399388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.399502 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.399928 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.399934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.399938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.400368 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.400504 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.400982 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.401202 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260806 08:57:48.401575 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.385896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55820)
0806 08:57:48.386807 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.386847 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.386994 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.387383 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.387389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.388284 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.388617 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.388637 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.390027 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.390031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.391307 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.391334 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.392281 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.392285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.392407 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.392953 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.392976 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.395171 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.398643 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.398653 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.398660 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.399281 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.399639 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.399644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.399647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.399789 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.400512 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.400520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.400673 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.400837 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.401170 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":538,"threads_started":1}
I20260806 08:57:48.403321 32534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.61@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
I20260806 08:57:48.414543 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.404037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42210)
0806 08:57:48.404224 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.404241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.404327 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.404549 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.404552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.404869 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.404998 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.405006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.405936 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.405940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.406794 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.406806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.407431 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.407437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.407642 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.408091 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.408107 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.409919 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.412563 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.412567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.412569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.412911 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.413275 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.413279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.413282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.413414 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.413857 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.413861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.413971 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.414084 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.414307 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 08:57:48.414799 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.404038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.22.98.61:36891)
0806 08:57:48.404176 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.404180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.404296 (+ 116us) server_negotiation.cc:407] Connection header received
0806 08:57:48.404399 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.404403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.404454 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.404526 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.405108 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.405801 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.406913 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.407308 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.407410 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.410104 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.410109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.410112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.410148 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.412451 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.413030 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.413035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.413037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.413173 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.413499 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.413502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.413505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.413766 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.413870 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.414181 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.414440 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":45}
I20260806 08:57:48.419425 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Bootstrap starting.
I20260806 08:57:48.422683 32534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:48.423810 32534 log.cc:824] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:48.426730 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: No bootstrap required, opened a new log
I20260806 08:57:48.433887 32534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } }
I20260806 08:57:48.434201 32534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:48.434309 32534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1011f202e44fb3a56b2197c6f8f2b1, State: Initialized, Role: FOLLOWER
I20260806 08:57:48.434886 32534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } }
I20260806 08:57:48.435772 32547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } } }
I20260806 08:57:48.436084 32547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:48.436748 32534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:48.442246 32547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:48.442535 32547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } }
I20260806 08:57:48.444027 32547 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807)
I20260806 08:57:48.445194 31827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e1a719411b4d1f907a0147a3870384" is_pre_election: true
W20260806 08:57:48.446028 32559 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 08:57:48.446151 32559 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 08:57:48.446421 32467 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1; no voters: b7e1a719411b4d1f907a0147a3870384
I20260806 08:57:48.446914 32547 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 08:57:48.450368 32441 master_runner.cc:428] Detected that this master 0b1011f202e44fb3a56b2197c6f8f2b1 is joining an existing cluster
I20260806 08:57:48.450485 32441 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.98.61:36891
I20260806 08:57:48.481648 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.459660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.22.98.61:36891)
0806 08:57:48.459836 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.459841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.466337 (+ 6496us) server_negotiation.cc:407] Connection header received
0806 08:57:48.466576 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.466582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.466671 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.466783 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.468287 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.469431 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.471295 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.471905 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.471993 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.474846 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.474853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.474857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.474890 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.478484 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.479294 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.479298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.479302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.479442 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.479984 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.479989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.479993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.480369 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.480506 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.481050 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.481280 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 08:57:48.482993 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.458938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42218)
0806 08:57:48.466177 (+ 7239us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.466205 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.466408 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.466842 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.466847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.467663 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.468112 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.468126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.469635 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.469641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.471158 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.471174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.472279 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.472284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.472448 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.473591 (+ 1143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.473613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.474627 (+ 1014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.478659 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.478663 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.478667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.479185 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.479598 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.479602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.479606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.479870 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.480597 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.480602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.480780 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.482229 (+ 1449us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.482541 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7035,"thread_start_us":9926,"threads_started":1}
WARNING: no policy specified for oryx/127.22.98.60@KRBTEST.COM; defaulting to no policy
I20260806 08:57:48.485230 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.458896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.22.98.62:40807)
0806 08:57:48.459467 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.459474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.468856 (+ 9382us) server_negotiation.cc:407] Connection header received
0806 08:57:48.469342 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.469348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.469445 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.469517 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.471116 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.472227 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.474262 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.474938 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.475041 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.477891 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.477899 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.477904 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.477946 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.481609 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.482416 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.482422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.482426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.482585 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.483129 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.483135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.483138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.483760 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.483974 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.484642 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.484890 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432}
I20260806 08:57:48.486377 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.459351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55832)
0806 08:57:48.468743 (+ 9392us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.468763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.469165 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.469774 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.469779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.470532 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.470900 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.470912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.472435 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.472440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.474081 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.474097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.475484 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.475493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.475736 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.476294 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.476321 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.477524 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.481787 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.481792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.481796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.482258 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.482758 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.482763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.482766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.482956 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.484084 (+ 1128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.484090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.484300 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.485506 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.485870 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9223,"thread_start_us":6451,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.60@KRBTEST.COM" created.
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006666, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.62@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
I20260806 08:57:48.520368 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.492742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42226)
0806 08:57:48.501296 (+ 8554us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.501350 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.501601 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.502458 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.502468 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.503491 (+ 1023us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.503884 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.503901 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.505405 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.505411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.507085 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.507104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.508123 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.508130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.508347 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.509203 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.509231 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.512438 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.517085 (+ 4647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.517103 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.517112 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.517666 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.518198 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.518204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.518208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.518393 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.519094 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.519104 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.519355 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.519578 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.520005 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8251,"thread_start_us":9095,"threads_started":1}
I20260806 08:57:48.520447 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.492746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.22.98.61:36891)
0806 08:57:48.492922 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.492929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.501516 (+ 8587us) server_negotiation.cc:407] Connection header received
0806 08:57:48.501911 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.501918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.502027 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.502163 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.504115 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.505240 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.507274 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.507922 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.508058 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.512680 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.512688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.512694 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.512745 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.516875 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.517816 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.517823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.517827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.518058 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.518509 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.518515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.518519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.518936 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.519144 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.519741 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.520018 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.60.keytab.
Entry for principal oryx/127.22.98.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.60.keytab.
I20260806 08:57:48.522521 31814 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.98.61:36891: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
member_type: NON_VOTER
last_known_addr {
host: "127.22.98.61"
port: 36891
}
attrs {
promote: true
}
}
dest_uuid: "b7e1a719411b4d1f907a0147a3870384"
cas_config_opid_index: -1
I20260806 08:57:48.523944 31814 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } }
I20260806 08:57:48.524307 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:35281
--webserver_interface=127.22.98.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.60.keytab
--principal=oryx/127.22.98.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:57:48.538223 32575 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } }
I20260806 08:57:48.540330 32500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:48.544090 32500 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 08:57:48.551769 32575 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.98.61:36891
I20260806 08:57:48.551968 32577 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:48.553108 32441 master.cc:537] Master@127.22.98.61:36891 shutting down...
I20260806 08:57:48.554390 32574 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } }
I20260806 08:57:48.554836 32574 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
W20260806 08:57:48.568430 31796 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 08:57:48.571871 32441 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:48.572286 32441 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:48.572458 32441 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: stopping tablet replica
I20260806 08:57:48.580490 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.579996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42240)
0806 08:57:48.580170 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.580297 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:36891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 08:57:48.580677 32441 master.cc:552] Master@127.22.98.61:36891 shutdown complete.
I20260806 08:57:48.580816 32441 master_runner.cc:305] Clearing existing system tablet
W20260806 08:57:48.581090 31796 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 -> Peer 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61:36891): Couldn't send request to peer 0b1011f202e44fb3a56b2197c6f8f2b1. Status: Network error: Client connection negotiation failed: client connection to 127.22.98.61:36891: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 08:57:48.581477 32441 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:48.581566 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:48.583768 32441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 08:57:48.585175 32582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:48.585451 32441 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:48.585731 32441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 08:57:48.585821 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "ca7a936155573f69c6bd46762930b857"
server_key_iv: "2e27fe76ce6f4417bbc3729767ab3c8b"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.589586 32441 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:48.594537 32441 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 08:57:48.594720 32441 log.cc:1196] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 08:57:48.595105 32441 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Deleting consensus metadata
I20260806 08:57:48.595350 32441 master_runner.cc:315] Copying system tablet from 127.22.98.62:40807
I20260806 08:57:48.597903 32441 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: tablet copy: Beginning tablet copy session from remote peer at address 127.22.98.62:40807
I20260806 08:57:48.610479 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.598900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.22.98.62:40807)
0806 08:57:48.599106 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:48.599113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:48.599576 (+ 463us) server_negotiation.cc:407] Connection header received
0806 08:57:48.599810 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:48.599814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:48.599879 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:48.599961 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:48.600810 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.601541 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.602764 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.603189 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.603299 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.605068 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:48.605074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:48.605077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:48.605108 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:48.607601 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.608406 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.608413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.608416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.608573 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:48.608995 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:48.609000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:48.609004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:48.609294 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:48.609435 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:48.609844 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 08:57:48.610055 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 08:57:48.610502 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:48.598821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55846)
0806 08:57:48.599449 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:48.599465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:48.599625 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:48.600010 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:48.600015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:48.600446 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:48.600657 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.600666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.601680 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.601688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.602622 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:48.602634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:48.603345 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:48.603351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:48.603511 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:48.604113 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:48.604130 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:48.604877 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:48.607775 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.607789 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.607793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.608233 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.608740 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:48.608745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:48.608747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:48.608888 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:48.609416 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:48.609421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:48.609569 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 08:57:48.609713 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 08:57:48.610065 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":404,"threads_started":1}
I20260806 08:57:48.611747 31843 tablet_copy_service.cc:140] P b7e1a719411b4d1f907a0147a3870384: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0b1011f202e44fb3a56b2197c6f8f2b1 ({username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:55846)
I20260806 08:57:48.612051 31843 tablet_copy_service.cc:161] P b7e1a719411b4d1f907a0147a3870384: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0b1011f202e44fb3a56b2197c6f8f2b1 at {username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:55846: session id = 0b1011f202e44fb3a56b2197c6f8f2b1-00000000000000000000000000000000
I20260806 08:57:48.615495 31843 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 08:57:48.617935 32441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 08:57:48.622789 32441 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: tablet copy: Starting download of 0 data blocks...
I20260806 08:57:48.623157 32441 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: tablet copy: Starting download of 1 WAL segments...
I20260806 08:57:48.626418 32441 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 08:57:48.630085 31843 tablet_copy_service.cc:342] P b7e1a719411b4d1f907a0147a3870384: Request end of tablet copy session 0b1011f202e44fb3a56b2197c6f8f2b1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:55846
I20260806 08:57:48.630308 31843 tablet_copy_service.cc:433] P b7e1a719411b4d1f907a0147a3870384: ending tablet copy session 0b1011f202e44fb3a56b2197c6f8f2b1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0b1011f202e44fb3a56b2197c6f8f2b1
I20260806 08:57:48.641613 32441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:48.644130 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.654354 32594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.656064 32591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:48.656656 32441 server_base.cc:1034] running on GCE node
I20260806 08:57:48.657021 32441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:48.657686 32441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:48.658928 32441 hybrid_clock.cc:648] HybridClock initialized: now 1786006668658866 us; error 36 us; skew 500 ppm
I20260806 08:57:48.659274 32441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:48.661168 32441 init.cc:377] Logged in from keytab as oryx/127.22.98.61@KRBTEST.COM (short username oryx)
I20260806 08:57:48.662393 32441 webserver.cc:460] Webserver started at http://127.22.98.61:34583/ using document root <none> and password file <none>
I20260806 08:57:48.662714 32441 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:48.662793 32441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:48.665145 32441 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 08:57:48.666817 32603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:48.667212 32441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 08:57:48.667330 32441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "ca7a936155573f69c6bd46762930b857"
server_key_iv: "2e27fe76ce6f4417bbc3729767ab3c8b"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.667475 32441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:48.678784 32441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:48.682014 32441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:48.706702 32441 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.61:36891
I20260806 08:57:48.706725 32681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.61:36891 every 8 connection(s)
I20260806 08:57:48.707901 32441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
I20260806 08:57:48.713673 32682 sys_catalog.cc:263] Verifying existing consensus state
I20260806 08:57:48.717420 32682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Bootstrap starting.
I20260806 08:57:48.738541 32682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 08:57:48.739048 32682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Bootstrap complete.
I20260806 08:57:48.740120 32682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } }
I20260806 08:57:48.740361 32682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0b1011f202e44fb3a56b2197c6f8f2b1, State: Initialized, Role: LEARNER
I20260806 08:57:48.740651 32682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } }
I20260806 08:57:48.741672 32682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:48.741458 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } }
I20260806 08:57:48.741940 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: LEARNER
I20260806 08:57:48.751443 32699 catalog_manager.cc:1259] Loaded cluster ID: 0d9ac1548a2d4b29a063d8f9ab8eadec
I20260806 08:57:48.751583 32699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: loading cluster ID for follower catalog manager: success
I20260806 08:57:48.755218 32699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: acquiring CA information for follower catalog manager: success
I20260806 08:57:48.757997 32699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 08:57:48.860792 32429 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
W20260806 08:57:48.942867 32576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:48.943198 32576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:48.943302 32576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:48.951437 32576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:48.951627 32576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:48.951705 32576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:48.951768 32576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:48.951813 32576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:48.962204 32576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.60.keytab
--principal=oryx/127.22.98.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:35281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.98.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:48.963558 32576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:48.964861 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:48.977622 32706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.977845 32707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:48.977869 32709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:48.978993 32576 server_base.cc:1034] running on GCE node
I20260806 08:57:48.979885 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:48.981320 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:48.982564 32576 hybrid_clock.cc:648] HybridClock initialized: now 1786006668982498 us; error 50 us; skew 500 ppm
I20260806 08:57:48.983002 32576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:48 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:48.986617 32576 init.cc:377] Logged in from keytab as oryx/127.22.98.60@KRBTEST.COM (short username oryx)
I20260806 08:57:48.988229 32576 webserver.cc:460] Webserver started at http://127.22.98.60:45149/ using document root <none> and password file <none>
I20260806 08:57:48.988901 32576 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:48.989003 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:48.989254 32576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:48.992272 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/instance:
uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "5c8e4f5ad828ee0319acf1d87a53d0be"
server_key_iv: "f0611cffef24e6c982b3fdd2e7ee5def"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.992988 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal/instance:
uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "5c8e4f5ad828ee0319acf1d87a53d0be"
server_key_iv: "f0611cffef24e6c982b3fdd2e7ee5def"
server_key_version: "encryptionkey@0"
I20260806 08:57:48.998348 32576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:57:49.001937 32717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.003302 32576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 08:57:49.003469 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "5c8e4f5ad828ee0319acf1d87a53d0be"
server_key_iv: "f0611cffef24e6c982b3fdd2e7ee5def"
server_key_version: "encryptionkey@0"
I20260806 08:57:49.003690 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:49.016387 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:49.020725 32576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:49.020992 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:49.041877 32576 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.60:35281
I20260806 08:57:49.041991 316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.60:35281 every 8 connection(s)
I20260806 08:57:49.043474 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
I20260806 08:57:49.048000 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 32576
I20260806 08:57:49.048067 317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:49.058218 317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.60@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:49.073864 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.050469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.22.98.60:35281)
0806 08:57:49.051756 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.051769 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.051821 (+ 52us) server_negotiation.cc:407] Connection header received
0806 08:57:49.053455 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.053480 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.053872 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.054226 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.055915 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.057464 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.058807 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.059720 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.059883 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.065286 (+ 5403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.065304 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.065320 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.065374 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.069329 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.070217 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.070227 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.070237 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.070464 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.070952 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.070956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.070958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.071604 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.071942 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.072399 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.072825 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":682,"threads_started":1}
I20260806 08:57:49.077132 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.060882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.22.98.62:40807)
0806 08:57:49.061113 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.061120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.061920 (+ 800us) server_negotiation.cc:407] Connection header received
0806 08:57:49.062110 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.062117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.062202 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.062324 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.063452 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.064565 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.066769 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.067384 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.067656 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.071493 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.071499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.071502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.071565 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.074337 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.075023 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.075029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.075032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.075214 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.075689 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.075693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.075695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.075996 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.076121 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.076624 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.076795 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 08:57:49.077289 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.060777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55858)
0806 08:57:49.061773 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.061816 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.061987 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.062368 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.062372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.063025 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.063296 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.063305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.064737 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.064771 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.066583 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.066600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.067727 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.067738 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.067933 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.068575 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.068603 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.071298 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.074461 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.074469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.074474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.074904 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.075336 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.075339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.075342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.075483 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.076123 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.076130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.076314 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.076543 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.076866 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":591,"threads_started":1}
I20260806 08:57:49.078815 317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.60@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
I20260806 08:57:49.088265 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.081547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42254)
0806 08:57:49.081967 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.081986 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.082093 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.082751 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.082756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.082783 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:49.083044 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.083058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.084746 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.084752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.086391 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.086409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.087593 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.087601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.087741 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.087786 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.087866 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.087968 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":215,"threads_started":1}
I20260806 08:57:49.088497 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.081884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.22.98.61:36891)
0806 08:57:49.082428 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.082432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.082451 (+ 19us) server_negotiation.cc:407] Connection header received
0806 08:57:49.082503 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.082506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.082585 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.082679 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:49.083239 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.084535 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.086549 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.087340 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.087442 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.087707 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.087912 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.088053 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":291,"threads_started":1}
I20260806 08:57:49.089840 32647 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 08:57:49.090489 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e1a719411b4d1f907a0147a3870384. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } }
I20260806 08:57:49.090761 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: LEARNER
I20260806 08:57:49.094521 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.079776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.22.98.61:36891)
0806 08:57:49.080335 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.080340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.080359 (+ 19us) server_negotiation.cc:407] Connection header received
0806 08:57:49.080420 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.080425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.080502 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.080614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.081371 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.082913 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.084854 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.085247 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.085354 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.089186 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.089193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.089196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.089236 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.092168 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.092935 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.092939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.092942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.093079 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.093408 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.093411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.093413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.093665 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.093799 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.094085 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.094238 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":318,"threads_started":1}
I20260806 08:57:49.094604 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.079596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42248)
0806 08:57:49.079804 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.079823 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.079916 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.080669 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.080673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.081049 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.081214 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.081222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.083073 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.083077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.084704 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.084730 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.085414 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.085420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.085613 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.086181 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.086203 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.088955 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.092294 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.092300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.092304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.092794 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.093180 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.093183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.093186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.093324 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.093748 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.093750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.093901 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.094000 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.094295 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 08:57:49.095616 317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006668, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.60@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
I20260806 08:57:49.108635 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.096515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.22.98.60:35281)
0806 08:57:49.096650 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.096654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.096669 (+ 15us) server_negotiation.cc:407] Connection header received
0806 08:57:49.096704 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.096707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.096760 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.096831 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.097489 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.098197 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.099262 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.099776 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.099883 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.103181 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.103186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.103189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.103218 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.105753 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.106499 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.106504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.106506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.106654 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.107093 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.107097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.107100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.107444 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.107602 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.108014 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.108350 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":49}
I20260806 08:57:49.108765 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.096242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:57314)
0806 08:57:49.096484 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.096507 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.096600 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.096853 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.096855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.097216 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.097378 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.097386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.098318 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.098322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.099121 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.099132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.099969 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.099974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.100164 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.100742 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.100759 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.102998 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.105886 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.105892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.105896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.106369 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.106765 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.106770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.106774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.106951 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.107580 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.107585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.107799 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.107935 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.108196 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 08:57:49.115152 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Bootstrap starting.
I20260806 08:57:49.119781 317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:49.121042 317 log.cc:824] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:49.123927 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: No bootstrap required, opened a new log
I20260806 08:57:49.132370 317 raft_consensus.cc:348] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } }
I20260806 08:57:49.132699 317 raft_consensus.cc:374] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:49.132800 317 raft_consensus.cc:729] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64bd62eac374ce49881a7eb9cbedd9f, State: Initialized, Role: FOLLOWER
I20260806 08:57:49.133299 317 consensus_queue.cc:260] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } }
I20260806 08:57:49.134116 334 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } } }
I20260806 08:57:49.134455 334 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:49.135118 317 sys_catalog.cc:564] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: configured and running, proceeding with master startup.
W20260806 08:57:49.145642 355 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 08:57:49.145778 355 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 08:57:49.149610 32576 master_runner.cc:428] Detected that this master c64bd62eac374ce49881a7eb9cbedd9f is joining an existing cluster
I20260806 08:57:49.149724 32576 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.98.60:35281
I20260806 08:57:49.173377 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.154187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.22.98.62:40807)
0806 08:57:49.154395 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.154402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.155248 (+ 846us) server_negotiation.cc:407] Connection header received
0806 08:57:49.155626 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.155632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.155717 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.155794 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.157740 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.159033 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.161585 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.162272 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.162389 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.165845 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.165854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.165860 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.165917 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.169997 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.170836 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.170843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.170847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.171002 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.171507 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.171512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.171515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.171891 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.172026 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.172742 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.172982 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 08:57:49.175181 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.154291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55862)
0806 08:57:49.155149 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.155168 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.155427 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.156035 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.156040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.156863 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.157544 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.157560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.159522 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.159528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.161397 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.161413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.162717 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.162724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.162917 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.163934 (+ 1017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.163961 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.165574 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.170196 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.170201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.170205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.170696 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.171154 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.171159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.171175 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.171369 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.172106 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.172110 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.172396 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.174054 (+ 1658us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.174410 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":36,"thread_start_us":13500,"threads_started":1}
I20260806 08:57:49.181116 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31896
I20260806 08:57:49.189209 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.155391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.22.98.60:35281)
0806 08:57:49.155586 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.155591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.168519 (+ 12928us) server_negotiation.cc:407] Connection header received
0806 08:57:49.168698 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.168703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.168774 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.168838 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.171963 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.173313 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.175817 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.176447 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.176570 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.179046 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.179054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.179059 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.179096 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.185920 (+ 6824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.186771 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.186776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.186780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.186918 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.187303 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.187309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.187312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.187837 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.188009 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.188501 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.188766 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":54}
I20260806 08:57:49.190924 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.155685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:57318)
0806 08:57:49.168439 (+ 12754us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.168455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.168597 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.169100 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.169104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.170040 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.170685 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.170700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.173695 (+ 2995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.173700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.175683 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.175699 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.176699 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.176707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.177202 (+ 495us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.177751 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.177768 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.178773 (+ 1005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.186078 (+ 7305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.186084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.186088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.186647 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.187042 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.187047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.187050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.187202 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.188076 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.188081 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.188251 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.190074 (+ 1823us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.190392 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12644,"thread_start_us":12518,"threads_started":1}
I20260806 08:57:49.197695 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.154414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.22.98.61:36891)
0806 08:57:49.154581 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.154587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.168278 (+ 13691us) server_negotiation.cc:407] Connection header received
0806 08:57:49.168557 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.168563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.168653 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.168771 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.170375 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.171402 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.173637 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.174299 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.174404 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.181775 (+ 7371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.181783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.181796 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.181842 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.187258 (+ 5416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.194427 (+ 7169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.194433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.194437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.194619 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.195459 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.195465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.195468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.195880 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.196020 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.197080 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.197346 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006666, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.62@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
I20260806 08:57:49.198984 362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.154367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:42268)
0806 08:57:49.168135 (+ 13768us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.168159 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.168394 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.168954 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.168959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.169804 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.170197 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.170208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.171636 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.171641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.173457 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.173473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.174881 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.174892 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.175103 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.180078 (+ 4975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.180107 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.181511 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.193186 (+ 11675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.193191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.193195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.193652 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.194803 (+ 1151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.194807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.194811 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.195006 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.196120 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.196125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.196353 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.198162 (+ 1809us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.198504 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13597,"mutex_wait_us":64,"thread_start_us":492,"threads_started":1}
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.1@KRBTEST.COM
I20260806 08:57:49.209722 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.184999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:57324)
0806 08:57:49.185175 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.185193 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.185315 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.190020 (+ 4705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.190026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.190645 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.190874 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.190889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.192737 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.192744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.194131 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.194150 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.195106 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.195113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.195307 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.196049 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.196074 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.201331 (+ 5257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.206394 (+ 5063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.206399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.206401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.207082 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.207604 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.207610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.207614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.207800 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.208640 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.208645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.208845 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.209009 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.209325 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 08:57:49.210901 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.185340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.22.98.60:35281)
0806 08:57:49.189510 (+ 4170us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.189515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.189532 (+ 17us) server_negotiation.cc:407] Connection header received
0806 08:57:49.189669 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.189675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.189796 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.189903 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.191069 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.192548 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.194344 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.194927 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.195050 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.201562 (+ 6512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.201569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.201573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.201623 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.205676 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.207237 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.207243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.207246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.207413 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.207950 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.207956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.207960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.208414 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.208555 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.210237 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.210501 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4081,"thread_start_us":17399,"threads_started":1}
I20260806 08:57:49.212688 31814 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.98.60:35281: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
member_type: NON_VOTER
last_known_addr {
host: "127.22.98.60"
port: 35281
}
attrs {
promote: true
}
}
dest_uuid: "b7e1a719411b4d1f907a0147a3870384"
cas_config_opid_index: 5
I20260806 08:57:49.213841 31814 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } }
I20260806 08:57:49.215842 325 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.219082 366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.183137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.22.98.1:32931)
0806 08:57:49.198842 (+ 15705us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.198854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.198887 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:57:49.198966 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.198975 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.199308 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.201718 (+ 2410us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.202008 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.203687 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.205058 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.205685 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.205812 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.210682 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.210695 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.210712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.210745 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.214122 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.214883 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.214893 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.214896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.215052 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.215589 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.215594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.215598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.216277 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.218058 (+ 1781us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.218280 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.218613 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15468,"thread_start_us":16311,"threads_started":1}
I20260806 08:57:49.220537 32647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 08:57:49.220753 32049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:49.221203 32752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:49.221961 325 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.98.60:35281
I20260806 08:57:49.229503 370 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 08:57:49.230674 325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.231134 325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:49.242431 32647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.248284 32752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 08:57:49.249112 32576 master.cc:537] Master@127.22.98.60:35281 shutting down...
I20260806 08:57:49.249347 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: NON_VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: true } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.249684 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: LEARNER
I20260806 08:57:49.250236 325 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:49.251401 370 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: attempting to promote NON_VOTER 0b1011f202e44fb3a56b2197c6f8f2b1 to VOTER
I20260806 08:57:49.252405 370 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } }
I20260806 08:57:49.265671 32576 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:49.266212 32576 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:49.266530 32576 tablet_replica.cc:330] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: stopping tablet replica
I20260806 08:57:49.271426 32647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 LEARNER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260806 08:57:49.272104 31796 connection.cc:698] client connection to 127.22.98.60:35281 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260806 08:57:49.272908 371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 08:57:49.275224 32576 master.cc:552] Master@127.22.98.60:35281 shutdown complete.
I20260806 08:57:49.275367 32576 master_runner.cc:305] Clearing existing system tablet
I20260806 08:57:49.275820 32576 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:49.276013 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:49.276396 370 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.280651 325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.280970 325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:49.281739 371 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.282045 371 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:49.282979 32647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.285683 32576 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.007s
I20260806 08:57:49.287350 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.287652 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:49.289535 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.289062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:57336)
0806 08:57:49.289225 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.289323 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:35281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 08:57:49.289855 31796 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 -> Peer c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60:35281): Couldn't send request to peer c64bd62eac374ce49881a7eb9cbedd9f. Status: Network error: Client connection negotiation failed: client connection to 127.22.98.60:35281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 08:57:49.295851 379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.296236 32576 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:49.296526 32576 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.009s
I20260806 08:57:49.296661 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "5c8e4f5ad828ee0319acf1d87a53d0be"
server_key_iv: "f0611cffef24e6c982b3fdd2e7ee5def"
server_key_version: "encryptionkey@0"
I20260806 08:57:49.300570 32576 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:49.305855 32576 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 08:57:49.306018 32576 log.cc:1196] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 08:57:49.306541 32576 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Deleting consensus metadata
I20260806 08:57:49.306783 32576 master_runner.cc:315] Copying system tablet from 127.22.98.62:40807
I20260806 08:57:49.309468 32576 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: tablet copy: Beginning tablet copy session from remote peer at address 127.22.98.62:40807
I20260806 08:57:49.322845 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.310241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.22.98.62:40807)
0806 08:57:49.310493 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.310501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.311028 (+ 527us) server_negotiation.cc:407] Connection header received
0806 08:57:49.311231 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.311237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.311320 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.311412 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.312687 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.313492 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.314774 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.315194 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.315315 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.317172 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.317178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.317181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.317222 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.320071 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.320713 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.320718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.320721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.320853 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.321348 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.321352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.321355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.321640 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.321787 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.322263 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.322489 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 08:57:49.322918 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.310184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:55878)
0806 08:57:49.310913 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.310930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.311078 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.311460 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.311465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.312229 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.312535 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.312546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.313639 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.313644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.314615 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.314628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.315347 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.315353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.315525 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.316178 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.316197 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.316950 (+ 753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.320199 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.320211 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.320220 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.320595 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.320963 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.320968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.320972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.321233 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.321765 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.321770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.321977 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.322122 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.322468 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":511,"threads_started":1}
I20260806 08:57:49.323796 31843 tablet_copy_service.cc:140] P b7e1a719411b4d1f907a0147a3870384: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c64bd62eac374ce49881a7eb9cbedd9f ({username='oryx', principal='oryx/127.22.98.60@KRBTEST.COM'} at 127.0.0.1:55878)
I20260806 08:57:49.323963 31843 tablet_copy_service.cc:161] P b7e1a719411b4d1f907a0147a3870384: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c64bd62eac374ce49881a7eb9cbedd9f at {username='oryx', principal='oryx/127.22.98.60@KRBTEST.COM'} at 127.0.0.1:55878: session id = c64bd62eac374ce49881a7eb9cbedd9f-00000000000000000000000000000000
I20260806 08:57:49.326681 31843 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 08:57:49.328413 32576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 08:57:49.333833 32576 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: tablet copy: Starting download of 0 data blocks...
I20260806 08:57:49.334199 32576 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: tablet copy: Starting download of 1 WAL segments...
I20260806 08:57:49.336613 32576 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 08:57:49.340046 31843 tablet_copy_service.cc:342] P b7e1a719411b4d1f907a0147a3870384: Request end of tablet copy session c64bd62eac374ce49881a7eb9cbedd9f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.98.60@KRBTEST.COM'} at 127.0.0.1:55878
I20260806 08:57:49.340175 31843 tablet_copy_service.cc:433] P b7e1a719411b4d1f907a0147a3870384: ending tablet copy session c64bd62eac374ce49881a7eb9cbedd9f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c64bd62eac374ce49881a7eb9cbedd9f
I20260806 08:57:49.350595 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:49.353253 386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:49.353958 387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:49.380709 389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:49.382208 32576 server_base.cc:1034] running on GCE node
I20260806 08:57:49.382627 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:49.383507 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:49.384830 32576 hybrid_clock.cc:648] HybridClock initialized: now 1786006669384801 us; error 39 us; skew 500 ppm
I20260806 08:57:49.385155 32576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006669, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:49.387753 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31896
I20260806 08:57:49.387858 32576 init.cc:377] Logged in from keytab as oryx/127.22.98.60@KRBTEST.COM (short username oryx)
I20260806 08:57:49.401160 32576 webserver.cc:460] Webserver started at http://127.22.98.60:38693/ using document root <none> and password file <none>
I20260806 08:57:49.401629 32576 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:49.401859 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:49.405802 32576 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:49.412175 397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.412683 32576 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 08:57:49.412866 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
format_stamp: "Formatted at 2026-08-06 08:57:48 on dist-test-slave-jpj1"
server_key: "5c8e4f5ad828ee0319acf1d87a53d0be"
server_key_iv: "f0611cffef24e6c982b3fdd2e7ee5def"
server_key_version: "encryptionkey@0"
I20260806 08:57:49.413128 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:49.415467 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:32931
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:40807
--webserver_port=44409
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:57:49.424506 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:49.428579 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:49.466395 32576 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.60:35281
I20260806 08:57:49.466460 460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.60:35281 every 8 connection(s)
I20260806 08:57:49.467654 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
I20260806 08:57:49.473101 461 sys_catalog.cc:263] Verifying existing consensus state
I20260806 08:57:49.477054 461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Bootstrap starting.
I20260806 08:57:49.499642 461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 08:57:49.500212 461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: Bootstrap complete.
I20260806 08:57:49.501329 461 raft_consensus.cc:348] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } }
I20260806 08:57:49.501657 461 raft_consensus.cc:729] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c64bd62eac374ce49881a7eb9cbedd9f, State: Initialized, Role: LEARNER
I20260806 08:57:49.501976 461 consensus_queue.cc:260] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } }
I20260806 08:57:49.503170 461 sys_catalog.cc:564] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:49.502907 465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.503576 465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: LEARNER
I20260806 08:57:49.510969 479 catalog_manager.cc:1259] Loaded cluster ID: 0d9ac1548a2d4b29a063d8f9ab8eadec
I20260806 08:57:49.511123 479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: loading cluster ID for follower catalog manager: success
I20260806 08:57:49.514683 479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: acquiring CA information for follower catalog manager: success
I20260806 08:57:49.517177 479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 08:57:49.804927 403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:49.805265 403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:49.805382 403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:49.813557 403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:49.813692 403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:49.813839 403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:49.824190 403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:32931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=44409
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:49.825553 403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:49.827006 403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:49.841809 490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:49.841859 495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:49.842044 489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:49.845064 403 server_base.cc:1034] running on GCE node
I20260806 08:57:49.845780 403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:49.846819 403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:49.848022 403 hybrid_clock.cc:648] HybridClock initialized: now 1786006669847967 us; error 49 us; skew 500 ppm
I20260806 08:57:49.848354 403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006669, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:49.851855 403 init.cc:377] Logged in from keytab as oryx/127.22.98.1@KRBTEST.COM (short username oryx)
I20260806 08:57:49.853675 403 webserver.cc:460] Webserver started at http://127.22.98.1:44409/ using document root <none> and password file <none>
I20260806 08:57:49.854701 403 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:49.854882 403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:49.861040 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.853320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:57346)
0806 08:57:49.854267 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.854288 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.854443 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.854817 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.854822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.854855 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:49.855197 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.855215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.856737 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.856744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.858658 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.858676 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.860227 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.860234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.860441 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.860478 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.860585 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.860737 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":703,"threads_started":1}
I20260806 08:57:49.861238 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.853568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.22.98.60:35281)
0806 08:57:49.854190 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.854196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.854365 (+ 169us) server_negotiation.cc:407] Connection header received
0806 08:57:49.854581 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.854584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.854658 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.854759 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:49.855350 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.856540 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.858851 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.859999 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.860174 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.860456 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.860599 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.860774 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":238,"threads_started":1}
I20260806 08:57:49.861683 403 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 08:57:49.862116 430 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 08:57:49.862664 465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e1a719411b4d1f907a0147a3870384. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:49.862951 465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: LEARNER
I20260806 08:57:49.865061 510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.866613 403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 08:57:49.866803 403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "4f72a334b34340958f1ba0d3171f79a6"
format_stamp: "Formatted at 2026-08-06 08:57:46 on dist-test-slave-jpj1"
server_key: "9b17cede0e5400029d9902b54a3a9963"
server_key_iv: "8a0e972cd677393ec5cbc378120c8c33"
server_key_version: "encryptionkey@0"
I20260806 08:57:49.867923 403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:49.888785 403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:49.893422 403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:49.893674 403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:49.894691 403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:49.896297 403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:49.896385 403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.896467 403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:49.896576 403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:49.933018 403 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:32931
I20260806 08:57:49.933094 657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:32931 every 8 connection(s)
I20260806 08:57:49.934499 403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:49.940645 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 403
I20260806 08:57:49.940989 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 32070
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006669, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006669, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006669, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
Aug 06 08:57:49 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.2@KRBTEST.COM
I20260806 08:57:49.963388 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.938001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:39953 (local address 127.22.98.60:35281)
0806 08:57:49.938456 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.938463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.939931 (+ 1468us) server_negotiation.cc:407] Connection header received
0806 08:57:49.943641 (+ 3710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.943646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.943727 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.943801 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.946013 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947125 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950240 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.950678 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950790 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.956047 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.956054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.956058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.956099 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.959483 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.960532 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.960539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.960543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.960713 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.961230 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.961236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.961240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.961544 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.961681 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.962777 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.963019 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260806 08:57:49.964782 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.938425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.22.98.1:39953)
0806 08:57:49.939734 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.939804 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.942742 (+ 2938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.944035 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.944039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.944566 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.945856 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.945869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947719 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.947727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.950065 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.950081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.950971 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950982 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.951397 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.952365 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.952405 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.955828 (+ 3423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.959776 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.959785 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.959803 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.960388 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.960855 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.960860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.960864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.961121 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.961779 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.961788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.962294 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.962628 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.963214 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":287,"spinlock_wait_cycles":111232,"thread_start_us":573,"threads_started":1}
I20260806 08:57:49.965549 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.940355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:44863 (local address 127.22.98.61:36891)
0806 08:57:49.940869 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.940876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.942866 (+ 1990us) server_negotiation.cc:407] Connection header received
0806 08:57:49.943168 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.943174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.943268 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.943364 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.945729 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947025 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950259 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.950962 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.951068 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.957822 (+ 6754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.957830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.957834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.957879 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.961392 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.962431 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.962437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.962441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.962668 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.963197 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.963202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.963204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.963685 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.963827 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.964951 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.965173 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":315,"threads_started":1}
I20260806 08:57:49.966198 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.940526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.22.98.1:44863)
0806 08:57:49.942047 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.942065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.942959 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.943562 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.943566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.945198 (+ 1632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.945556 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.945569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947213 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.947216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.950082 (+ 2866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.950097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.951346 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.951352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.951676 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.952855 (+ 1179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.952880 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.956654 (+ 3774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.961536 (+ 4882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.961540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.961543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.962281 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.962793 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.962798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.962801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.963017 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.963889 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.963893 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.964036 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.964231 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.964503 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1392,"mutex_wait_us":281,"thread_start_us":1226,"threads_started":1}
I20260806 08:57:49.966619 658 heartbeater.cc:347] Connected to a master server at 127.22.98.60:35281
I20260806 08:57:49.967027 658 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:49.967330 660 heartbeater.cc:347] Connected to a master server at 127.22.98.61:36891
I20260806 08:57:49.967475 660 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:49.968146 658 heartbeater.cc:511] Master 127.22.98.60:35281 requested a full tablet report, sending...
I20260806 08:57:49.968173 660 heartbeater.cc:511] Master 127.22.98.61:36891 requested a full tablet report, sending...
I20260806 08:57:49.968598 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.937690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:55557 (local address 127.22.98.62:40807)
0806 08:57:49.938463 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.938469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.941570 (+ 3101us) server_negotiation.cc:407] Connection header received
0806 08:57:49.943038 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.943044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.943134 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.943241 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.945697 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947011 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950208 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.950861 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.950983 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.955793 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.955801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.955806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.955850 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.960029 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.963221 (+ 3192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.963228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.963231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.963386 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.964522 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.964528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.964531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.965204 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.965349 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.966510 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.968141 (+ 1631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":486,"threads_started":1}
I20260806 08:57:49.968853 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.942365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.22.98.2:39299)
0806 08:57:49.942981 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:49.942990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:49.943018 (+ 28us) server_negotiation.cc:407] Connection header received
0806 08:57:49.943085 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:49.943093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:49.943399 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:49.943574 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:49.945999 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947458 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.949326 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.951212 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.951332 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.959386 (+ 8054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:49.959403 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:49.959423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:49.959473 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:49.963049 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.964501 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.964510 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.964513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.964705 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:49.965154 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:49.965159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:49.965162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:49.965555 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:49.965841 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:49.967631 (+ 1790us) server_negotiation.cc:299] Negotiation successful
0806 08:57:49.967909 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260806 08:57:49.969491 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:49.938425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:55557)
0806 08:57:49.939734 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:49.939804 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:49.942747 (+ 2943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:49.943523 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:49.943558 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:49.944486 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:49.945510 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.945541 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.947338 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.947343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.950081 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:49.950092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:49.951148 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:49.951156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:49.951435 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:49.952367 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:49.952397 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:49.955463 (+ 3066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:49.961570 (+ 6107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.961575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.961578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.962285 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.963819 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:49.963824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:49.963827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:49.964015 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:49.965904 (+ 1889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:49.965908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:49.966024 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 08:57:49.966151 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 08:57:49.966441 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":455,"spinlock_wait_cycles":8960,"thread_start_us":605,"threads_started":1}
I20260806 08:57:49.970240 32232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:49.970613 659 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:49.970582 419 ts_manager.cc:194] Registered new tserver with Master: 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:49.970582 32630 ts_manager.cc:194] Registered new tserver with Master: 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:49.970742 659 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:49.971210 659 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:49.972434 31814 ts_manager.cc:194] Re-registered known tserver with Master: 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:49.973259 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:55557
I20260806 08:57:50.058736 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 32070
I20260806 08:57:50.081019 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:39299
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:40807
--webserver_port=42667
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:57:50.286830 370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: NON_VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: true } } }
I20260806 08:57:50.287199 370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:50.288609 370 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: attempting to promote NON_VOTER c64bd62eac374ce49881a7eb9cbedd9f to VOTER
I20260806 08:57:50.289197 370 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:50.291368 32647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 08:57:50.291652 430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 LEARNER]: Refusing update from remote peer b7e1a719411b4d1f907a0147a3870384: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 08:57:50.292107 679 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 08:57:50.292543 678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 08:57:50.296821 679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.297664 430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.300963 679 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.301178 678 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.301301 679 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:50.301517 678 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:50.303129 465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.303428 465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:50.309636 678 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.309970 678 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:50.310654 32647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.313860 685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e1a719411b4d1f907a0147a3870384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:50.314234 685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 08:57:50.424417 677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:50.424760 677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:50.424883 677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:50.432226 677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:50.432343 677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:50.432466 677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:50.442571 677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:39299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=42667
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:50.444016 677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:50.445457 677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:50.459002 693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:50.459013 696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:50.459225 694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:50.459947 677 server_base.cc:1034] running on GCE node
I20260806 08:57:50.460553 677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:50.461475 677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:50.462669 677 hybrid_clock.cc:648] HybridClock initialized: now 1786006670462607 us; error 54 us; skew 500 ppm
I20260806 08:57:50.462972 677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:50 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006670, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:50.466651 677 init.cc:377] Logged in from keytab as oryx/127.22.98.2@KRBTEST.COM (short username oryx)
I20260806 08:57:50.468380 677 webserver.cc:460] Webserver started at http://127.22.98.2:42667/ using document root <none> and password file <none>
I20260806 08:57:50.469206 677 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:50.469388 677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:50.476195 677 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 08:57:50.479508 703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:50.481082 677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:57:50.481253 677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "675f008bf969f4c62ba0ce7b301fc3b0"
server_key_iv: "fd41fb54e228e5a0c537297940786159"
server_key_version: "encryptionkey@0"
I20260806 08:57:50.482549 677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:50.500602 677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:50.504794 677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:50.505069 677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:50.506057 677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:50.507452 677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:50.507568 677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:50.507678 677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:50.507741 677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:50.540499 677 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39299
I20260806 08:57:50.540571 855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39299 every 8 connection(s)
I20260806 08:57:50.542003 677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:50.546123 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 677
I20260806 08:57:50.546439 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 32249
Aug 06 08:57:50 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006670, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
I20260806 08:57:50.584411 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.545448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:41893 (local address 127.22.98.61:36891)
0806 08:57:50.557824 (+ 12376us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:50.557832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:50.558768 (+ 936us) server_negotiation.cc:407] Connection header received
0806 08:57:50.561911 (+ 3143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:50.561919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:50.562035 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:50.562424 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:50.565205 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.566425 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.569356 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.570177 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.570282 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.575950 (+ 5668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:50.575958 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:50.575962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:50.576020 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:50.579649 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.580768 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.580774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.580777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.580938 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.581541 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.581547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.581550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.581904 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:50.582063 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:50.583798 (+ 1735us) server_negotiation.cc:299] Negotiation successful
0806 08:57:50.584036 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12231,"thread_start_us":12043,"threads_started":1}
I20260806 08:57:50.585180 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.545750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.22.98.2:41893)
0806 08:57:50.558488 (+ 12738us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:50.558633 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:50.561508 (+ 2875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:50.563091 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:50.563104 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:50.564053 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:50.564968 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.564996 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.566679 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.566685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.569178 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.569196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.570735 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.570743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.571065 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.571921 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:50.571963 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:50.575664 (+ 3701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:50.579809 (+ 4145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.579819 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.579836 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.580601 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.581079 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.581084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.581088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.581400 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.582147 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:50.582156 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:50.582616 (+ 460us) client_negotiation.cc:769] Sending connection context
0806 08:57:50.582949 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 08:57:50.583558 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12415,"thread_start_us":12172,"threads_started":1}
Aug 06 08:57:50 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.3@KRBTEST.COM
Aug 06 08:57:50 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006670, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
Aug 06 08:57:50 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006670, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:50.587062 860 heartbeater.cc:347] Connected to a master server at 127.22.98.61:36891
I20260806 08:57:50.587404 860 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:50.588464 860 heartbeater.cc:511] Master 127.22.98.61:36891 requested a full tablet report, sending...
I20260806 08:57:50.590467 32630 ts_manager.cc:194] Registered new tserver with Master: 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299)
I20260806 08:57:50.595067 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.559368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:34733 (local address 127.22.98.60:35281)
0806 08:57:50.577396 (+ 18028us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:50.577400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:50.577417 (+ 17us) server_negotiation.cc:407] Connection header received
0806 08:57:50.577463 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:50.577466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:50.577520 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:50.577627 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:50.578575 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.579735 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.581814 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.582512 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.582635 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.587816 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:50.587826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:50.587831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:50.587874 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:50.592440 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.593195 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.593201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.593204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.593345 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.593691 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.593696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.593700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.593962 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:50.594090 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:50.594448 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 08:57:50.594695 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17922,"thread_start_us":17779,"threads_started":1}
I20260806 08:57:50.595175 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.565386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.22.98.2:34733)
0806 08:57:50.576816 (+ 11430us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:50.576841 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:50.576940 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:50.577679 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:50.577684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:50.578131 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:50.578380 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.578391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.579874 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.579878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.581641 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.581656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.582934 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.582942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.583124 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.583798 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:50.583819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:50.586861 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:50.592580 (+ 5719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.592585 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.592589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.593083 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.593452 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.593457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.593460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.593588 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.594074 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:50.594079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:50.594198 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 08:57:50.594331 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 08:57:50.594637 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10826,"mutex_wait_us":63,"thread_start_us":6364,"threads_started":1}
I20260806 08:57:50.595890 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.547725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.22.98.3:35517)
0806 08:57:50.568427 (+ 20702us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:50.568437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:50.571945 (+ 3508us) server_negotiation.cc:407] Connection header received
0806 08:57:50.572141 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:50.572151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:50.572545 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:50.572713 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:50.575142 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.576849 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.578944 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.579919 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.580040 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.587761 (+ 7721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:50.587775 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:50.587790 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:50.587827 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:50.591419 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.592198 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.592207 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.592211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.592355 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.592813 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.592819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.592823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.593341 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:50.593648 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:50.595156 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0806 08:57:50.595511 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20492,"thread_start_us":20710,"threads_started":1}
I20260806 08:57:50.595992 856 heartbeater.cc:347] Connected to a master server at 127.22.98.60:35281
I20260806 08:57:50.596139 856 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:50.596515 856 heartbeater.cc:511] Master 127.22.98.60:35281 requested a full tablet report, sending...
I20260806 08:57:50.597371 32422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:50.597785 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.545750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:44779)
0806 08:57:50.569123 (+ 23373us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:50.569150 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:50.569279 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:50.575176 (+ 5897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:50.575180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:50.576129 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:50.576664 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.576677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.579855 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.579861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.581707 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:50.581718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.582766 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.582773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:50.583338 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.585160 (+ 1822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:50.585185 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:50.589548 (+ 4363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:50.594293 (+ 4745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.594299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.594302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.594914 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.595823 (+ 909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:50.595828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:50.595831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:50.596116 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:50.596764 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:50.596769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:50.596900 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 08:57:50.597028 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 08:57:50.597309 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23210,"mutex_wait_us":56,"spinlock_wait_cycles":71552,"thread_start_us":23439,"threads_started":1}
I20260806 08:57:50.597846 419 ts_manager.cc:194] Registered new tserver with Master: 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299)
I20260806 08:57:50.598062 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:50.545367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:44779 (local address 127.22.98.62:40807)
0806 08:57:50.574119 (+ 28752us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:50.574126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:50.574147 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:57:50.574213 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:50.574218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:50.574294 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:50.574406 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:50.578483 (+ 4077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.579694 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.581855 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:50.582509 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:50.582930 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:50.589830 (+ 6900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:50.589837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:50.589841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:50.589886 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:50.594166 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.595311 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.595317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.595321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.595485 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:50.596279 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:50.596285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:50.596288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:50.596624 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:50.596761 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:50.597351 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 08:57:50.597578 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28616,"thread_start_us":29508,"threads_started":1}
I20260806 08:57:50.598953 857 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:50.599071 857 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:50.599437 857 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:50.600394 31814 ts_manager.cc:194] Re-registered known tserver with Master: 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299)
I20260806 08:57:50.601253 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:44779
I20260806 08:57:50.691108 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 32249
I20260806 08:57:50.710872 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:35517
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:40807
--webserver_port=34855
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:57:50.976091 659 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
W20260806 08:57:51.057648 874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:51.057978 874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:51.058133 874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:51.065491 874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:51.065629 874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:51.065790 874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:51.076195 874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:35517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=34855
--tserver_master_addrs=127.22.98.62:40807,127.22.98.61:36891,127.22.98.60:35281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:51.077661 874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:51.079051 874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:51.093178 883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:51.093132 886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:51.093096 884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:51.094009 874 server_base.cc:1034] running on GCE node
I20260806 08:57:51.094671 874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:51.096046 874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:51.097282 874 hybrid_clock.cc:648] HybridClock initialized: now 1786006671097219 us; error 55 us; skew 500 ppm
I20260806 08:57:51.097620 874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:51 dist-test-slave-jpj1 krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006671, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:51.101037 874 init.cc:377] Logged in from keytab as oryx/127.22.98.3@KRBTEST.COM (short username oryx)
I20260806 08:57:51.102510 874 webserver.cc:460] Webserver started at http://127.22.98.3:34855/ using document root <none> and password file <none>
I20260806 08:57:51.103292 874 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:51.103435 874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:51.109377 874 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 08:57:51.112484 895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:51.113938 874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:51.114130 874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "4cf77181c0df4751b893340a4d0f37a8"
format_stamp: "Formatted at 2026-08-06 08:57:47 on dist-test-slave-jpj1"
server_key: "6ea07da81ee713ac6f50841abe7e3824"
server_key_iv: "2b3453ed0ec4bb20b1273a6054a4321e"
server_key_version: "encryptionkey@0"
I20260806 08:57:51.115145 874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:51.126204 874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:51.129959 874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:51.130232 874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:51.131177 874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:57:51.132539 874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:51.132637 874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:51.132750 874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:51.132822 874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:51.163527 874 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:35517
I20260806 08:57:51.163643 1058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:35517 every 8 connection(s)
I20260806 08:57:51.164986 874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:51.166976 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 874
Aug 06 08:57:51 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006671, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.61@KRBTEST.COM
Aug 06 08:57:51 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006671, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
Aug 06 08:57:51 dist-test-slave-jpj1 krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006671, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.60@KRBTEST.COM
I20260806 08:57:51.220002 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.168707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:34211 (local address 127.22.98.62:40807)
0806 08:57:51.169235 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.169243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.176245 (+ 7002us) server_negotiation.cc:407] Connection header received
0806 08:57:51.176440 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.176445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.176535 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.176661 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.183956 (+ 7295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.185147 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.192780 (+ 7633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.193495 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.193639 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.206411 (+ 12772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.206420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.206424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.206477 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.213074 (+ 6597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.214396 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.214403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.214406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.214580 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.215187 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.215192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.215196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.215944 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.216095 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.219300 (+ 3205us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.219603 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":282,"threads_started":1}
I20260806 08:57:51.221966 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.168702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:34211)
0806 08:57:51.176113 (+ 7411us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.176140 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.176292 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.177573 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.177579 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.182704 (+ 5125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:51.183730 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.183763 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.191094 (+ 7331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.191101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.192605 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.192623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.195715 (+ 3092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.195724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.195927 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.199302 (+ 3375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:51.199334 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:51.206168 (+ 6834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:51.213237 (+ 7069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.213241 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.213245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.213866 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.214739 (+ 873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.214745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.214749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.215034 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.216183 (+ 1149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:51.216193 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:51.216737 (+ 544us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.217126 (+ 389us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.217911 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7256,"thread_start_us":654,"threads_started":1}
I20260806 08:57:51.222956 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.168638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.22.98.3:52855)
0806 08:57:51.169906 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.169961 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.173153 (+ 3192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.177389 (+ 4236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.177404 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.183996 (+ 6592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:51.184264 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.184275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.186561 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.186568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.189759 (+ 3191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.189778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.194262 (+ 4484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.194271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.194609 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.196854 (+ 2245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:51.196892 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:51.205759 (+ 8867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:51.212748 (+ 6989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.212760 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.212797 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.213642 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.220653 (+ 7011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.220658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.220662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.221003 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.221853 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:51.221859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:51.222051 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.222196 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.222512 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":6991,"threads_started":1}
I20260806 08:57:51.223966 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.169071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:52855 (local address 127.22.98.61:36891)
0806 08:57:51.176760 (+ 7689us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.176768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.176789 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:57:51.176853 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.176859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.176936 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.177049 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.184451 (+ 7402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.186302 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.189956 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.190604 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.190746 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.206464 (+ 15718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.206473 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.206478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.206539 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.210910 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.214019 (+ 3109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.214026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.214029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.214212 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.221153 (+ 6941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.221159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.221163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.221562 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.221779 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.223292 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.223578 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7536,"thread_start_us":7641,"threads_started":1}
I20260806 08:57:51.224193 1062 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40807
I20260806 08:57:51.224556 1062 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:51.224944 1063 heartbeater.cc:347] Connected to a master server at 127.22.98.61:36891
I20260806 08:57:51.225097 1063 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:51.225857 1062 heartbeater.cc:511] Master 127.22.98.62:40807 requested a full tablet report, sending...
I20260806 08:57:51.225970 1063 heartbeater.cc:511] Master 127.22.98.61:36891 requested a full tablet report, sending...
I20260806 08:57:51.228019 31814 ts_manager.cc:194] Re-registered known tserver with Master: 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:51.228377 32630 ts_manager.cc:194] Registered new tserver with Master: 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:51.229198 31814 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:34211
I20260806 08:57:51.231155 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.179523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.22.98.62:40807)
0806 08:57:51.204260 (+ 24737us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.204267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.204301 (+ 34us) server_negotiation.cc:407] Connection header received
0806 08:57:51.204376 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.204382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.204468 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.204593 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.208965 (+ 4372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.210326 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.218384 (+ 8058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.219244 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.219349 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.222945 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.222953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.222958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.223001 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.226959 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.228145 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.228151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.228154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.228342 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.228882 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.228888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.228891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.229295 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.229529 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.230528 (+ 999us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.230857 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24571,"thread_start_us":25212,"threads_started":1}
I20260806 08:57:51.235456 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.179166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.22.98.61:36891)
0806 08:57:51.192092 (+ 12926us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.192098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.193229 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 08:57:51.193295 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.193301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.193387 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.193484 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.207641 (+ 14157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.208957 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.210979 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.211711 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.211850 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.218056 (+ 6206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.218063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.218068 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.218110 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.231374 (+ 13264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.232302 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.232309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.232320 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.232478 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.232986 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.232991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.232995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.233399 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.233658 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.234876 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.235121 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12777,"thread_start_us":12922,"threads_started":1}
I20260806 08:57:51.238729 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.176418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:49403 (local address 127.22.98.60:35281)
0806 08:57:51.201984 (+ 25566us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.201992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.204155 (+ 2163us) server_negotiation.cc:407] Connection header received
0806 08:57:51.207200 (+ 3045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.207206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.207295 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.207395 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.210540 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.211875 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.214433 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.214900 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.215020 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.220112 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.220119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.220124 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.220166 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.235031 (+ 14865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.236242 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.236248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.236251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.236433 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.236957 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.236963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.236966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.237355 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.237526 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.238112 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.238353 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25408,"thread_start_us":25273,"threads_started":1}
I20260806 08:57:51.238749 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.178816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.22.98.3:49403)
0806 08:57:51.204002 (+ 25186us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.204025 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.204907 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.207663 (+ 2756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.207668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.208305 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:51.208696 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.208711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.212072 (+ 3361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.212079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.214230 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.214246 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.215091 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.215098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.215283 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.215957 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:51.215981 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:51.219828 (+ 3847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:51.235206 (+ 15378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.235212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.235216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.236076 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.236598 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:51.236604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:51.236607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:51.236814 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:51.237602 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:51.237607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:51.237774 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.237952 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.238333 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25044,"thread_start_us":24883,"threads_started":1}
I20260806 08:57:51.240850 1061 heartbeater.cc:347] Connected to a master server at 127.22.98.60:35281
I20260806 08:57:51.241071 1061 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:51.241711 1061 heartbeater.cc:511] Master 127.22.98.60:35281 requested a full tablet report, sending...
I20260806 08:57:51.243109 419 ts_manager.cc:194] Registered new tserver with Master: 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:51.243994 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.195471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.22.98.60:35281)
0806 08:57:51.207203 (+ 11732us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.207209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.214307 (+ 7098us) server_negotiation.cc:407] Connection header received
0806 08:57:51.215879 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.215885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.215988 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.216085 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.224508 (+ 8423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.225945 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.227974 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.228729 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.228838 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.237100 (+ 8262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.237107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.237112 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.237174 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.240607 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.241513 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.241518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.241521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.241682 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.242153 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.242157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.242160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.242607 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.242953 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.243303 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.243640 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11585,"spinlock_wait_cycles":896,"thread_start_us":11697,"threads_started":1}
I20260806 08:57:51.237370 31814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:51.247056 31814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:51.281193 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.267747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:32931 (local address 127.0.0.1:48490)
0806 08:57:51.268329 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.268350 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.268531 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.269478 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.269483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.269511 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:51.270036 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.270050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.271865 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.271874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.274303 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.274322 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.280097 (+ 5775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.280107 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.280325 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.280367 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.280449 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.280623 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":289,"threads_started":1}
I20260806 08:57:51.282079 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.269024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.22.98.2:39299)
0806 08:57:51.269862 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.269874 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.272183 (+ 2309us) server_negotiation.cc:407] Connection header received
0806 08:57:51.272533 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.272549 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.272866 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.273058 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:51.274070 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.275709 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.278437 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.280059 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.280170 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.280406 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.281113 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.281612 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":429,"threads_started":1}
I20260806 08:57:51.283195 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.268516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39299 (local address 127.0.0.1:44704)
0806 08:57:51.272056 (+ 3540us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.272082 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.272297 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.273343 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.273349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.273378 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:51.273845 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.273856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.275934 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.275940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.278225 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.278242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.280628 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.280635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.280879 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.280915 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.282615 (+ 1700us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.282802 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3399,"thread_start_us":3243,"threads_started":1}
I20260806 08:57:51.283203 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.268120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.22.98.1:32931)
0806 08:57:51.268729 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.268739 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.268769 (+ 30us) server_negotiation.cc:407] Connection header received
0806 08:57:51.268878 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.268888 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.269211 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.270224 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:51.270427 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.271715 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.277977 (+ 6262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.279783 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.282027 (+ 2244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.282351 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.282441 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.282661 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":369,"threads_started":1}
I20260806 08:57:51.287844 571 tablet_service.cc:1467] Processing CreateTablet for tablet a6cbcab872e841f1bca0c50514b4628c (DEFAULT_TABLE table=test-table [id=faade748e3024a12b848d32b8dfcaf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:51.288749 764 tablet_service.cc:1467] Processing CreateTablet for tablet a6cbcab872e841f1bca0c50514b4628c (DEFAULT_TABLE table=test-table [id=faade748e3024a12b848d32b8dfcaf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:51.289738 571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cbcab872e841f1bca0c50514b4628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:51.290190 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cbcab872e841f1bca0c50514b4628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:51.293279 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.272429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36228 (local address 127.22.98.3:35517)
0806 08:57:51.284488 (+ 12059us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.284497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.284535 (+ 38us) server_negotiation.cc:407] Connection header received
0806 08:57:51.284643 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.284652 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.284979 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.285199 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:51.285895 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.287743 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.290247 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.292181 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.292319 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.292546 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.292771 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.292952 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11858,"thread_start_us":11506,"threads_started":1}
I20260806 08:57:51.293517 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.272136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35517 (local address 127.0.0.1:36228)
0806 08:57:51.281560 (+ 9424us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.281582 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.281706 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.285273 (+ 3567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.285277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.285304 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:51.285516 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.285529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.287895 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.287901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.290056 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.290073 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.292347 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.292354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.292549 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.292586 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.292699 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.292863 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9243,"thread_start_us":12255,"threads_started":1}
I20260806 08:57:51.299484 959 tablet_service.cc:1467] Processing CreateTablet for tablet a6cbcab872e841f1bca0c50514b4628c (DEFAULT_TABLE table=test-table [id=faade748e3024a12b848d32b8dfcaf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:51.301093 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cbcab872e841f1bca0c50514b4628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:51.305078 1103 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Bootstrap starting.
I20260806 08:57:51.308447 1104 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Bootstrap starting.
I20260806 08:57:51.309268 1103 tablet_bootstrap.cc:654] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:51.310951 1103 log.cc:824] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:51.312301 1104 tablet_bootstrap.cc:654] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:51.313545 1104 log.cc:824] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:51.314778 1103 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: No bootstrap required, opened a new log
I20260806 08:57:51.315042 1103 ts_tablet_manager.cc:1397] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260806 08:57:51.316989 1104 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: No bootstrap required, opened a new log
I20260806 08:57:51.317271 1104 ts_tablet_manager.cc:1397] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260806 08:57:51.319466 1107 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Bootstrap starting.
I20260806 08:57:51.322644 1107 tablet_bootstrap.cc:654] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:51.323813 1107 log.cc:824] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:51.326273 1103 raft_consensus.cc:348] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.326767 1103 raft_consensus.cc:374] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:51.326861 1103 raft_consensus.cc:729] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f72a334b34340958f1ba0d3171f79a6, State: Initialized, Role: FOLLOWER
I20260806 08:57:51.327616 1107 tablet_bootstrap.cc:492] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: No bootstrap required, opened a new log
I20260806 08:57:51.327557 1103 consensus_queue.cc:260] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.327907 1107 ts_tablet_manager.cc:1397] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 08:57:51.329485 1103 ts_tablet_manager.cc:1428] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
I20260806 08:57:51.329839 1104 raft_consensus.cc:348] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.330317 1104 raft_consensus.cc:374] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:51.330420 1104 raft_consensus.cc:729] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d1c77a4ec0449dbc1dc6967b7f6ef3, State: Initialized, Role: FOLLOWER
I20260806 08:57:51.331216 1104 consensus_queue.cc:260] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.335136 857 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
I20260806 08:57:51.336984 1104 ts_tablet_manager.cc:1428] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Time spent starting tablet: real 0.020s user 0.018s sys 0.000s
I20260806 08:57:51.339872 1107 raft_consensus.cc:348] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.340404 1107 raft_consensus.cc:374] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:51.340514 1107 raft_consensus.cc:729] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf77181c0df4751b893340a4d0f37a8, State: Initialized, Role: FOLLOWER
I20260806 08:57:51.341183 1107 consensus_queue.cc:260] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.342446 1062 heartbeater.cc:503] Master 127.22.98.62:40807 was elected leader, sending a full tablet report...
I20260806 08:57:51.343290 1107 ts_tablet_manager.cc:1428] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Time spent starting tablet: real 0.015s user 0.002s sys 0.011s
W20260806 08:57:51.418236 1064 tablet.cc:2369] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:57:51.438418 663 tablet.cc:2369] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:51.515947 1113 raft_consensus.cc:482] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:51.516275 1113 raft_consensus.cc:504] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.519240 1113 leader_election.cc:288] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931), 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:51.526750 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.519674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:46997 (local address 127.22.98.3:35517)
0806 08:57:51.519875 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.519881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.520638 (+ 757us) server_negotiation.cc:407] Connection header received
0806 08:57:51.520811 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.520815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.520876 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.520955 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:51.521698 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.522456 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.524991 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.525760 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.525864 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.525948 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.526359 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.526498 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 08:57:51.526890 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.518769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:60319 (local address 127.22.98.1:32931)
0806 08:57:51.519038 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.519046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.520591 (+ 1545us) server_negotiation.cc:407] Connection header received
0806 08:57:51.520801 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.520807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.520872 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.520950 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:51.521722 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.522480 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.524800 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.525852 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.526003 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.526120 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.526512 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.526668 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 08:57:51.526998 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.519480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35517 (local address 127.22.98.2:46997)
0806 08:57:51.520508 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.520525 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.520667 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.520985 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.520989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.521036 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:51.521468 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.521478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.522639 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.522643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.524764 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.524781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.525964 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.525971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.526161 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.526227 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.526301 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.526446 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":22,"spinlock_wait_cycles":6912,"thread_start_us":658,"threads_started":1}
I20260806 08:57:51.527164 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.518697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:32931 (local address 127.22.98.2:60319)
0806 08:57:51.520455 (+ 1758us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:51.520473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:51.520646 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:51.520981 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:51.520986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:51.521022 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:51.521569 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.521579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.522612 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.522616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.524572 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:51.524587 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.526083 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.526088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:51.526252 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.526276 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 08:57:51.526369 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 08:57:51.526495 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1550,"mutex_wait_us":92,"spinlock_wait_cycles":3840,"thread_start_us":568,"threads_started":1}
I20260806 08:57:51.528044 993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cbcab872e841f1bca0c50514b4628c" candidate_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf77181c0df4751b893340a4d0f37a8" is_pre_election: true
I20260806 08:57:51.528188 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cbcab872e841f1bca0c50514b4628c" candidate_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f72a334b34340958f1ba0d3171f79a6" is_pre_election: true
I20260806 08:57:51.528592 993 raft_consensus.cc:2454] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38d1c77a4ec0449dbc1dc6967b7f6ef3 in term 0.
I20260806 08:57:51.528729 594 raft_consensus.cc:2454] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38d1c77a4ec0449dbc1dc6967b7f6ef3 in term 0.
I20260806 08:57:51.529263 706 leader_election.cc:302] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d1c77a4ec0449dbc1dc6967b7f6ef3, 4cf77181c0df4751b893340a4d0f37a8; no voters:
I20260806 08:57:51.529717 1113 raft_consensus.cc:2781] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:51.529989 1113 raft_consensus.cc:482] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:51.530119 1113 raft_consensus.cc:3037] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:51.532789 1113 raft_consensus.cc:504] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.533385 1113 leader_election.cc:288] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931), 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517)
I20260806 08:57:51.533876 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cbcab872e841f1bca0c50514b4628c" candidate_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f72a334b34340958f1ba0d3171f79a6"
I20260806 08:57:51.533913 993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cbcab872e841f1bca0c50514b4628c" candidate_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf77181c0df4751b893340a4d0f37a8"
I20260806 08:57:51.534075 594 raft_consensus.cc:3037] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:51.534102 993 raft_consensus.cc:3037] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:51.536527 993 raft_consensus.cc:2454] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38d1c77a4ec0449dbc1dc6967b7f6ef3 in term 1.
I20260806 08:57:51.536545 594 raft_consensus.cc:2454] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38d1c77a4ec0449dbc1dc6967b7f6ef3 in term 1.
I20260806 08:57:51.536965 706 leader_election.cc:302] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d1c77a4ec0449dbc1dc6967b7f6ef3, 4cf77181c0df4751b893340a4d0f37a8; no voters:
I20260806 08:57:51.537263 1113 raft_consensus.cc:2781] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:51.537739 1113 raft_consensus.cc:686] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 LEADER]: Becoming Leader. State: Replica: 38d1c77a4ec0449dbc1dc6967b7f6ef3, State: Running, Role: LEADER
I20260806 08:57:51.538358 1113 consensus_queue.cc:237] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:51.544997 31814 catalog_manager.cc:5495] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } health_report { overall_health: UNKNOWN } } }
W20260806 08:57:51.545894 862 tablet.cc:2369] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:51.594671 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.588929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.22.98.2:39299)
0806 08:57:51.589120 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.589125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.589144 (+ 19us) server_negotiation.cc:407] Connection header received
0806 08:57:51.589253 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.589257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.589368 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.589492 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:51.589927 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.590928 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.592434 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.592857 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.592976 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.593486 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:51.594022 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:51.594168 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.594310 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.594399 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 08:57:51.603602 594 raft_consensus.cc:1261] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Refusing update from remote peer 38d1c77a4ec0449dbc1dc6967b7f6ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:51.603838 993 raft_consensus.cc:1261] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Refusing update from remote peer 38d1c77a4ec0449dbc1dc6967b7f6ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:51.604676 1117 consensus_queue.cc:1035] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:51.605048 1113 consensus_queue.cc:1035] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:51.618511 1120 mvcc.cc:205] Tried to move back new op lower bound from 7315483326877057024 to 7315483326625497088. Current Snapshot: MvccSnapshot[applied={T|T < 7315483326877057024}]
I20260806 08:57:51.623780 1122 mvcc.cc:205] Tried to move back new op lower bound from 7315483326877057024 to 7315483326625497088. Current Snapshot: MvccSnapshot[applied={T|T < 7315483326877057024}]
I20260806 08:57:51.639268 31814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:51.639516 31814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:51.647246 31814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7e1a719411b4d1f907a0147a3870384: Sending DeleteTablet for 3 replicas of tablet a6cbcab872e841f1bca0c50514b4628c
I20260806 08:57:51.648663 571 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cbcab872e841f1bca0c50514b4628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:51 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:48490
I20260806 08:57:51.648990 764 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cbcab872e841f1bca0c50514b4628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:51 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:44704
I20260806 08:57:51.649075 959 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cbcab872e841f1bca0c50514b4628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:51 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:36228
I20260806 08:57:51.659878 1135 tablet_replica.cc:330] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: stopping tablet replica
I20260806 08:57:51.660176 1137 tablet_replica.cc:330] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: stopping tablet replica
I20260806 08:57:51.660466 1135 raft_consensus.cc:2229] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:51.660567 1137 raft_consensus.cc:2229] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:51.661038 1137 raft_consensus.cc:2258] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:51.661139 1135 raft_consensus.cc:2258] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:51.661954 1136 tablet_replica.cc:330] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: stopping tablet replica
I20260806 08:57:51.662376 1136 raft_consensus.cc:2229] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:51.663132 1136 raft_consensus.cc:2258] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:51.663931 1135 ts_tablet_manager.cc:1905] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:51.665292 1137 ts_tablet_manager.cc:1905] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:51.669615 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 31775
I20260806 08:57:51.670874 31855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:51.671763 1136 ts_tablet_manager.cc:1905] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:51.671814 1135 ts_tablet_manager.cc:1918] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:51.671989 1135 log.cc:1196] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/a6cbcab872e841f1bca0c50514b4628c
I20260806 08:57:51.672650 1135 ts_tablet_manager.cc:1939] T a6cbcab872e841f1bca0c50514b4628c P 4f72a334b34340958f1ba0d3171f79a6: Deleting consensus metadata
I20260806 08:57:51.679143 1137 ts_tablet_manager.cc:1918] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:51.679395 1137 log.cc:1196] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/a6cbcab872e841f1bca0c50514b4628c
I20260806 08:57:51.680020 1137 ts_tablet_manager.cc:1939] T a6cbcab872e841f1bca0c50514b4628c P 4cf77181c0df4751b893340a4d0f37a8: Deleting consensus metadata
I20260806 08:57:51.686288 1136 ts_tablet_manager.cc:1918] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:51.686502 1136 log.cc:1196] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/a6cbcab872e841f1bca0c50514b4628c
I20260806 08:57:51.687175 1136 ts_tablet_manager.cc:1939] T a6cbcab872e841f1bca0c50514b4628c P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting consensus metadata
I20260806 08:57:51.691252 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.652324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45890 (local address 127.22.98.61:36891)
0806 08:57:51.652520 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.652527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.667524 (+ 14997us) server_negotiation.cc:407] Connection header received
0806 08:57:51.668046 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.668052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.668143 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.668270 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.674316 (+ 6046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.675461 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.677384 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.678297 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.678446 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.682098 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.682106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.682110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.682161 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.686649 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.687625 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.687632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.687636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.687809 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.688686 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.688692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.688695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.689241 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.689493 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.690400 (+ 907us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.690659 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:57:51.714457 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.667837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.22.98.60:35281)
0806 08:57:51.668049 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.668055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.690830 (+ 22775us) server_negotiation.cc:407] Connection header received
0806 08:57:51.691010 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.691016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.691095 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.691187 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.693281 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.694283 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.696195 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.696843 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.696972 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.700601 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.700609 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.700614 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.700661 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.704691 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.705614 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.705619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.705624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.705831 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.711734 (+ 5903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.711742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.711746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.712206 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.712518 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.713844 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.714111 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 08:57:51.851969 31795 catalog_manager.cc:4841] TS 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931): tablet a6cbcab872e841f1bca0c50514b4628c (table test-table [id=faade748e3024a12b848d32b8dfcaf2c]) successfully deleted
I20260806 08:57:51.852667 31796 catalog_manager.cc:4841] TS 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299): tablet a6cbcab872e841f1bca0c50514b4628c (table test-table [id=faade748e3024a12b848d32b8dfcaf2c]) successfully deleted
I20260806 08:57:51.852775 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 31775
I20260806 08:57:51.900566 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.884414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45902 (local address 127.22.98.61:36891)
0806 08:57:51.884603 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.884609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.885715 (+ 1106us) server_negotiation.cc:407] Connection header received
0806 08:57:51.886068 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.886072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.886158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.886246 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.888247 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.889147 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.890401 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.890918 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.891037 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.894196 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.894202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.894206 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.894241 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.897168 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.898318 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.898324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.898328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.898486 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.898908 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.898913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.898917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.899291 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.899590 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.899947 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.900195 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 08:57:51.902217 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:51.885732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.22.98.60:35281)
0806 08:57:51.885921 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:51.885927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:51.886017 (+ 90us) server_negotiation.cc:407] Connection header received
0806 08:57:51.886172 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:51.886177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:51.886251 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:51.886341 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:51.888655 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.889877 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.891963 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:51.892572 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:51.892718 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:51.895870 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:51.895877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:51.895883 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:51.895925 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:51.898847 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.899697 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.899703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.899707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.899844 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:51.900286 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:51.900291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:51.900295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:51.900685 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:51.901012 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:51.901599 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 08:57:51.901837 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 08:57:52.629498 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:52.628602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:34269)
0806 08:57:52.629091 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:52.629269 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
W20260806 08:57:52.630182 857 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:52.630345 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:52.628797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:40173)
0806 08:57:52.630025 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:52.630158 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":1014,"threads_started":1}
W20260806 08:57:52.630697 659 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:52.636528 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:52.635785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:43793)
0806 08:57:52.636234 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:52.636352 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":228,"threads_started":1}
W20260806 08:57:52.636961 1062 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.147390 1154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7e1a719411b4d1f907a0147a3870384)
I20260806 08:57:53.147609 1157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7e1a719411b4d1f907a0147a3870384)
I20260806 08:57:53.147634 1154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:53.147912 1157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:53.148619 1154 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60:35281)
I20260806 08:57:53.149583 1157 leader_election.cc:288] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61:36891)
I20260806 08:57:53.156621 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.148786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33544)
0806 08:57:53.156038 (+ 7252us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.156300 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":431,"threads_started":1}
W20260806 08:57:53.157809 32604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.161581 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.149647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33548)
0806 08:57:53.161065 (+ 11418us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.161295 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11300,"threads_started":1}
I20260806 08:57:53.162493 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.161998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33560)
0806 08:57:53.162215 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.162316 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 08:57:53.162731 32604 leader_election.cc:334] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
W20260806 08:57:53.163129 398 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.165083 1161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.164481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33572)
0806 08:57:53.164693 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.164882 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260806 08:57:53.165361 398 leader_election.cc:334] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.169094 1160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.149089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.22.98.60:35281)
0806 08:57:53.161645 (+ 12556us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:53.161651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:53.161674 (+ 23us) server_negotiation.cc:407] Connection header received
0806 08:57:53.161752 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:53.161757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:53.161842 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:53.162696 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:53.162945 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.164268 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.166813 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.168102 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.168228 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:53.168336 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:53.168590 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 08:57:53.168724 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12387,"thread_start_us":12844,"threads_started":1}
I20260806 08:57:53.169132 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.148866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:35281 (local address 127.0.0.1:44872)
0806 08:57:53.159556 (+ 10690us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.159579 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:53.159740 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:53.162103 (+ 2363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:53.162108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:53.162144 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:53.162502 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:53.162519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.164435 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.164441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:53.166649 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:53.166666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.168277 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.168283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:53.168438 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:53.168463 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 08:57:53.168556 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 08:57:53.168689 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10554,"mutex_wait_us":21,"thread_start_us":10398,"threads_started":1}
I20260806 08:57:53.169482 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.150026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:36891 (local address 127.0.0.1:45918)
0806 08:57:53.161807 (+ 11781us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.161828 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:53.161973 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:53.163936 (+ 1963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:53.163942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:53.163968 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:53.164257 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:53.164271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.165932 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.165937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:53.167439 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:53.167454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.168769 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.168775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:53.168986 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:53.169020 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 08:57:53.169105 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 08:57:53.169257 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11686,"thread_start_us":10756,"threads_started":1}
I20260806 08:57:53.169849 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.159622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.22.98.61:36891)
0806 08:57:53.162569 (+ 2947us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:53.162575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:53.162600 (+ 25us) server_negotiation.cc:407] Connection header received
0806 08:57:53.162658 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:53.162663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:53.162745 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:53.162848 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:53.164476 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.165804 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.167646 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:53.168589 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:53.168712 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:53.168830 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:53.169153 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 08:57:53.169279 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":2833,"thread_start_us":2578,"threads_started":1}
I20260806 08:57:53.170240 430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" is_pre_election: true
I20260806 08:57:53.170372 32647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" is_pre_election: true
I20260806 08:57:53.170580 430 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1011f202e44fb3a56b2197c6f8f2b1 in term 1.
I20260806 08:57:53.170687 32647 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c64bd62eac374ce49881a7eb9cbedd9f in term 1.
I20260806 08:57:53.171041 32606 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1, c64bd62eac374ce49881a7eb9cbedd9f; no voters: b7e1a719411b4d1f907a0147a3870384
I20260806 08:57:53.171329 1154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 08:57:53.171445 1154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b7e1a719411b4d1f907a0147a3870384)
I20260806 08:57:53.171576 1154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 08:57:53.171602 400 leader_election.cc:302] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1, c64bd62eac374ce49881a7eb9cbedd9f; no voters: b7e1a719411b4d1f907a0147a3870384
I20260806 08:57:53.171991 1157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 08:57:53.172107 1157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Starting leader election (detected failure of leader b7e1a719411b4d1f907a0147a3870384)
I20260806 08:57:53.172206 1157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 1 FOLLOWER]: Advancing to term 2
I20260806 08:57:53.173991 1154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:53.174456 1157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:53.174628 1154 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 election: Requested vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60:35281)
I20260806 08:57:53.175172 1157 leader_election.cc:288] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 election: Requested vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), 0b1011f202e44fb3a56b2197c6f8f2b1 (127.22.98.61:36891)
I20260806 08:57:53.175201 430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
I20260806 08:57:53.175369 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.174801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33588)
0806 08:57:53.175133 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.175226 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260806 08:57:53.175591 430 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b1011f202e44fb3a56b2197c6f8f2b1 in current term 2: Already voted for candidate c64bd62eac374ce49881a7eb9cbedd9f in this term.
I20260806 08:57:53.175954 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.175488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33590)
0806 08:57:53.175692 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.175780 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 08:57:53.175937 32647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1"
I20260806 08:57:53.176234 32647 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c64bd62eac374ce49881a7eb9cbedd9f in current term 2: Already voted for candidate 0b1011f202e44fb3a56b2197c6f8f2b1 in this term.
I20260806 08:57:53.177016 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.176583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33598)
0806 08:57:53.176844 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.176908 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260806 08:57:53.177284 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.176925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33610)
0806 08:57:53.177075 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.177170 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 08:57:53.177255 398 leader_election.cc:334] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.177351 398 leader_election.cc:302] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c64bd62eac374ce49881a7eb9cbedd9f; no voters: 0b1011f202e44fb3a56b2197c6f8f2b1, b7e1a719411b4d1f907a0147a3870384
W20260806 08:57:53.177464 32604 leader_election.cc:334] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:53.177624 1157 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 08:57:53.177649 32604 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1; no voters: b7e1a719411b4d1f907a0147a3870384, c64bd62eac374ce49881a7eb9cbedd9f
I20260806 08:57:53.177919 1154 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 08:57:53.633526 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.631324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:35573)
0806 08:57:53.633180 (+ 1856us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.633290 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1721,"thread_start_us":1605,"threads_started":1}
I20260806 08:57:53.633731 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.630970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:46663)
0806 08:57:53.633438 (+ 2468us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.633538 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2331,"thread_start_us":2200,"threads_started":1}
I20260806 08:57:53.638403 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:53.637604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:58667)
0806 08:57:53.638091 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:53.638208 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":253,"threads_started":1}
I20260806 08:57:54.635648 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.634631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:45251)
0806 08:57:54.635363 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.635447 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":509,"threads_started":1}
I20260806 08:57:54.635609 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.634588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:46713)
0806 08:57:54.635288 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.635399 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":280,"threads_started":1}
I20260806 08:57:54.640045 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.639223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:41949)
0806 08:57:54.639725 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.639825 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":249,"threads_started":1}
I20260806 08:57:54.710865 1177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:54.711066 1177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:54.711795 1177 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60:35281)
I20260806 08:57:54.712378 430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" is_pre_election: true
I20260806 08:57:54.712647 430 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1011f202e44fb3a56b2197c6f8f2b1 in term 2.
I20260806 08:57:54.712870 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.711960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33830)
0806 08:57:54.712468 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.712604 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":230,"threads_started":1}
I20260806 08:57:54.713196 32606 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1, c64bd62eac374ce49881a7eb9cbedd9f; no voters:
I20260806 08:57:54.713511 1177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 08:57:54.713660 1177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:54.713771 1177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 2 FOLLOWER]: Advancing to term 3
I20260806 08:57:54.714483 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.714053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33844)
0806 08:57:54.714248 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.714336 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 08:57:54.714677 32604 leader_election.cc:334] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:54.716239 1177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:54.716765 1177 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 election: Requested vote from peers b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807), c64bd62eac374ce49881a7eb9cbedd9f (127.22.98.60:35281)
I20260806 08:57:54.717280 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.716909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33858)
0806 08:57:54.717072 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.717154 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 08:57:54.717366 430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c64bd62eac374ce49881a7eb9cbedd9f"
I20260806 08:57:54.717634 430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 2 FOLLOWER]: Advancing to term 3
I20260806 08:57:54.718119 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.717814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33866)
0806 08:57:54.717944 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.718018 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 08:57:54.718293 32604 leader_election.cc:334] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
I20260806 08:57:54.719970 430 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1011f202e44fb3a56b2197c6f8f2b1 in term 3.
I20260806 08:57:54.720367 32606 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1011f202e44fb3a56b2197c6f8f2b1, c64bd62eac374ce49881a7eb9cbedd9f; no voters: b7e1a719411b4d1f907a0147a3870384
I20260806 08:57:54.720619 1177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 08:57:54.720986 1177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [term 3 LEADER]: Becoming Leader. State: Replica: 0b1011f202e44fb3a56b2197c6f8f2b1, State: Running, Role: LEADER
I20260806 08:57:54.721357 1177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } }
I20260806 08:57:54.723286 1183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b1011f202e44fb3a56b2197c6f8f2b1. Latest consensus state: current_term: 3 leader_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:54.723664 1183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:54.724277 1185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:54.728410 1185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:54.729243 1185 catalog_manager.cc:1259] Loaded cluster ID: 0d9ac1548a2d4b29a063d8f9ab8eadec
I20260806 08:57:54.729321 1185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:54.730496 1185 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:54.731333 1185 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Loaded TSK: 0
I20260806 08:57:54.732259 1185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:57:54.788106 32629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:54.795112 32629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:54.799420 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.798966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33888)
0806 08:57:54.799113 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.799219 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 08:57:54.799630 430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [term 3 FOLLOWER]: Refusing update from remote peer 0b1011f202e44fb3a56b2197c6f8f2b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 08:57:54.800591 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.800108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33898)
0806 08:57:54.800367 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.800458 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 08:57:54.800696 1183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 08:57:54.801065 32604 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 -> Peer b7e1a719411b4d1f907a0147a3870384 (127.22.98.62:40807): Couldn't send request to peer b7e1a719411b4d1f907a0147a3870384. Status: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 08:57:54.804075 1187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b1011f202e44fb3a56b2197c6f8f2b1. Latest consensus state: current_term: 3 leader_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:54.804450 1187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:54.805682 1177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:54.806034 1177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:54.806739 1187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b7e1a719411b4d1f907a0147a3870384" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40807 } } peers { permanent_uuid: "0b1011f202e44fb3a56b2197c6f8f2b1" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 36891 } attrs { promote: false } } peers { permanent_uuid: "c64bd62eac374ce49881a7eb9cbedd9f" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 35281 } attrs { promote: false } } }
I20260806 08:57:54.807019 1187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64bd62eac374ce49881a7eb9cbedd9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 08:57:54.824383 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.814286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39299 (local address 127.0.0.1:44712)
0806 08:57:54.814529 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.814549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:54.814666 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:54.815623 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:54.815629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:54.815655 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:54.815999 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.816014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.818044 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.818051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.820249 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.820267 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.823421 (+ 3154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.823429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.823678 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.823724 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 08:57:54.823810 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 08:57:54.823973 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 08:57:54.827090 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.814381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.22.98.2:39299)
0806 08:57:54.815243 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.815250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.815271 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:57:54.815346 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.815351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.815431 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.816438 (+ 1007us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:54.816659 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.817864 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.821828 (+ 3964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.823184 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.826483 (+ 3299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.826591 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.826650 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.826812 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":587,"threads_started":1}
I20260806 08:57:54.827836 764 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (DEFAULT_TABLE table=test-table [id=447efa1804c346beba76ae575b99d525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:54.828285 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9047c03bd24bf0950ed7aa512fe8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:54.833783 1204 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Bootstrap starting.
I20260806 08:57:54.835848 1204 tablet_bootstrap.cc:654] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:54.836496 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.815244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:35517 (local address 127.0.0.1:36242)
0806 08:57:54.826580 (+ 11336us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.826609 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:54.826755 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:54.827490 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:54.827496 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:54.827524 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:54.827891 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.827903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.829604 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.829610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.831864 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.831883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.835609 (+ 3726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.835616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.835833 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.835870 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:57:54.835961 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 08:57:54.836124 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11201,"thread_start_us":11064,"threads_started":1}
I20260806 08:57:54.837333 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.815386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.22.98.3:35517)
0806 08:57:54.826145 (+ 10759us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.826152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.826958 (+ 806us) server_negotiation.cc:407] Connection header received
0806 08:57:54.827087 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.827094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.827179 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.827279 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:54.828189 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.829402 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.832811 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.835179 (+ 2368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.835325 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.835450 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.836855 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.837015 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10619,"thread_start_us":11461,"threads_started":1}
I20260806 08:57:54.838182 959 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (DEFAULT_TABLE table=test-table [id=447efa1804c346beba76ae575b99d525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:54.838699 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9047c03bd24bf0950ed7aa512fe8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:54.839324 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.814687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:32931 (local address 127.0.0.1:48496)
0806 08:57:54.824642 (+ 9955us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.824665 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:54.824790 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:54.832689 (+ 7899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:54.832694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:54.832723 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:54.832882 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.832894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.835365 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.835372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.837289 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.837306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.838622 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.838628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.838798 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.838825 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 08:57:54.838904 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 08:57:54.839035 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9844,"thread_start_us":12032,"threads_started":1}
I20260806 08:57:54.839380 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.815089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.22.98.1:32931)
0806 08:57:54.832388 (+ 17299us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.832393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.832414 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:57:54.832473 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.832477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.832547 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.832628 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:54.833070 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.835184 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.837535 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.838466 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.838589 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.838712 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.838945 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.839077 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17179,"thread_start_us":17063,"threads_started":1}
I20260806 08:57:54.840229 571 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (DEFAULT_TABLE table=test-table [id=447efa1804c346beba76ae575b99d525]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:54.840612 571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9047c03bd24bf0950ed7aa512fe8c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:54.844419 1204 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: No bootstrap required, opened a new log
I20260806 08:57:54.844579 1204 ts_tablet_manager.cc:1397] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260806 08:57:54.845465 1204 raft_consensus.cc:348] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.845781 1204 raft_consensus.cc:374] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:54.845911 1204 raft_consensus.cc:729] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d1c77a4ec0449dbc1dc6967b7f6ef3, State: Initialized, Role: FOLLOWER
I20260806 08:57:54.846160 1204 consensus_queue.cc:260] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.848394 1208 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Bootstrap starting.
I20260806 08:57:54.850165 1208 tablet_bootstrap.cc:654] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:54.850276 1204 ts_tablet_manager.cc:1428] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 08:57:54.851820 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.851223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:56665)
0806 08:57:54.851572 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.851671 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 08:57:54.853063 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.852579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:34373)
0806 08:57:54.852826 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.852914 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 08:57:54.854223 860 heartbeater.cc:503] Master 127.22.98.61:36891 was elected leader, sending a full tablet report...
I20260806 08:57:54.854823 1206 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Bootstrap starting.
I20260806 08:57:54.856422 1208 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: No bootstrap required, opened a new log
I20260806 08:57:54.856573 1208 ts_tablet_manager.cc:1397] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 08:57:54.856633 1206 tablet_bootstrap.cc:654] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:54.857437 1208 raft_consensus.cc:348] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.857720 1208 raft_consensus.cc:374] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:54.857836 1208 raft_consensus.cc:729] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f72a334b34340958f1ba0d3171f79a6, State: Initialized, Role: FOLLOWER
I20260806 08:57:54.858119 1208 consensus_queue.cc:260] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.859163 1206 tablet_bootstrap.cc:492] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: No bootstrap required, opened a new log
I20260806 08:57:54.859310 1206 ts_tablet_manager.cc:1397] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 08:57:54.860009 1208 ts_tablet_manager.cc:1428] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 08:57:54.860347 1206 raft_consensus.cc:348] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.860678 1206 raft_consensus.cc:374] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:54.860836 1206 raft_consensus.cc:729] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf77181c0df4751b893340a4d0f37a8, State: Initialized, Role: FOLLOWER
I20260806 08:57:54.861132 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.860573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:34659)
0806 08:57:54.860866 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.860967 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260806 08:57:54.861174 1206 consensus_queue.cc:260] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.862221 660 heartbeater.cc:503] Master 127.22.98.61:36891 was elected leader, sending a full tablet report...
I20260806 08:57:54.862385 1206 ts_tablet_manager.cc:1428] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 08:57:54.863131 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.862691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:50959)
0806 08:57:54.862885 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.862969 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 08:57:54.864148 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.863733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:33623)
0806 08:57:54.863928 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.864006 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 08:57:54.865176 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.864740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:46875)
0806 08:57:54.864916 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.865005 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 08:57:54.865844 1063 heartbeater.cc:503] Master 127.22.98.61:36891 was elected leader, sending a full tablet report...
I20260806 08:57:54.883677 1215 raft_consensus.cc:482] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:54.883956 1215 raft_consensus.cc:504] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.885601 1215 leader_election.cc:288] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299), 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:54.892956 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.885902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:55715 (local address 127.22.98.2:39299)
0806 08:57:54.886115 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.886122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.886224 (+ 102us) server_negotiation.cc:407] Connection header received
0806 08:57:54.886501 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.886505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.886566 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.886653 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:54.887157 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.888117 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.890648 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.891935 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.892074 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.892186 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.892571 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.892697 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:57:54.892976 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.886194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:53921 (local address 127.22.98.1:32931)
0806 08:57:54.886422 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.886429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.886828 (+ 399us) server_negotiation.cc:407] Connection header received
0806 08:57:54.887006 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.887010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.887077 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.887163 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:54.887625 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.888502 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.890835 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.892015 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.892137 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.892227 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.892551 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.892685 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 08:57:54.893055 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.885787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39299 (local address 127.22.98.3:55715)
0806 08:57:54.886116 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.886140 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:54.886289 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:54.886691 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:54.886696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:54.886727 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:54.886961 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.886971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.888270 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.888274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.890478 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.890494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.892123 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.892141 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.892375 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.892432 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 08:57:54.892519 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 08:57:54.892714 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":112}
I20260806 08:57:54.893656 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.886002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:32931 (local address 127.22.98.3:53921)
0806 08:57:54.886717 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.886733 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:54.886869 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:54.887191 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:54.887194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:54.887214 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:54.887460 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.887469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.888678 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.888684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.890666 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:54.890679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.892183 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.892188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:54.892380 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.892426 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 08:57:54.892523 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 08:57:54.892732 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":28,"thread_start_us":396,"threads_started":1}
I20260806 08:57:54.893949 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9047c03bd24bf0950ed7aa512fe8c6" candidate_uuid: "4cf77181c0df4751b893340a4d0f37a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" is_pre_election: true
I20260806 08:57:54.894349 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9047c03bd24bf0950ed7aa512fe8c6" candidate_uuid: "4cf77181c0df4751b893340a4d0f37a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f72a334b34340958f1ba0d3171f79a6" is_pre_election: true
I20260806 08:57:54.894411 790 raft_consensus.cc:2454] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf77181c0df4751b893340a4d0f37a8 in term 0.
I20260806 08:57:54.894565 594 raft_consensus.cc:2454] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf77181c0df4751b893340a4d0f37a8 in term 0.
I20260806 08:57:54.895030 898 leader_election.cc:302] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d1c77a4ec0449dbc1dc6967b7f6ef3, 4cf77181c0df4751b893340a4d0f37a8; no voters:
I20260806 08:57:54.895392 1215 raft_consensus.cc:2781] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:54.895516 1215 raft_consensus.cc:482] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:54.895676 1215 raft_consensus.cc:3037] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:54.898366 1215 raft_consensus.cc:504] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.898983 1215 leader_election.cc:288] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [CANDIDATE]: Term 1 election: Requested vote from peers 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299), 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931)
I20260806 08:57:54.899467 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9047c03bd24bf0950ed7aa512fe8c6" candidate_uuid: "4cf77181c0df4751b893340a4d0f37a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3"
I20260806 08:57:54.899468 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9047c03bd24bf0950ed7aa512fe8c6" candidate_uuid: "4cf77181c0df4751b893340a4d0f37a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f72a334b34340958f1ba0d3171f79a6"
I20260806 08:57:54.899691 790 raft_consensus.cc:3037] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:54.899700 594 raft_consensus.cc:3037] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:54.901690 594 raft_consensus.cc:2454] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf77181c0df4751b893340a4d0f37a8 in term 1.
I20260806 08:57:54.901787 790 raft_consensus.cc:2454] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf77181c0df4751b893340a4d0f37a8 in term 1.
I20260806 08:57:54.902130 897 leader_election.cc:302] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf77181c0df4751b893340a4d0f37a8, 4f72a334b34340958f1ba0d3171f79a6; no voters:
I20260806 08:57:54.902454 1215 raft_consensus.cc:2781] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:54.902900 1215 raft_consensus.cc:686] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf77181c0df4751b893340a4d0f37a8, State: Running, Role: LEADER
I20260806 08:57:54.903330 1215 consensus_queue.cc:237] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } }
I20260806 08:57:54.904083 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.903492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:46639)
0806 08:57:54.903800 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.903918 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 08:57:54.907226 32629 catalog_manager.cc:5495] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "4cf77181c0df4751b893340a4d0f37a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf77181c0df4751b893340a4d0f37a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 35517 } health_report { overall_health: HEALTHY } } }
I20260806 08:57:54.928545 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.923451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.22.98.3:35517)
0806 08:57:54.923648 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.923653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.923678 (+ 25us) server_negotiation.cc:407] Connection header received
0806 08:57:54.923884 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.923888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.923973 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.924097 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:54.924460 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.925195 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.926496 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:54.926950 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:54.927045 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:54.927383 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:54.928047 (+ 664us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:54.928159 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:54.928250 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 08:57:54.928319 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 08:57:54.932945 594 raft_consensus.cc:1261] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Refusing update from remote peer 4cf77181c0df4751b893340a4d0f37a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:54.933358 790 raft_consensus.cc:1261] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Refusing update from remote peer 4cf77181c0df4751b893340a4d0f37a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:54.933704 1217 consensus_queue.cc:1035] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f72a334b34340958f1ba0d3171f79a6" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:54.934108 1215 consensus_queue.cc:1035] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38d1c77a4ec0449dbc1dc6967b7f6ef3" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:54.936550 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.936024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:57277)
0806 08:57:54.936292 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.936395 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 08:57:54.937323 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.936866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:51327)
0806 08:57:54.937090 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.937176 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 08:57:54.948189 1230 mvcc.cc:205] Tried to move back new op lower bound from 7315483340517486592 to 7315483340408758272. Current Snapshot: MvccSnapshot[applied={T|T < 7315483340517486592}]
I20260806 08:57:54.949678 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.949114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.1:57279)
0806 08:57:54.949365 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.949457 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 08:57:54.951154 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.948482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:41513)
0806 08:57:54.950858 (+ 2376us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.950974 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2253}
I20260806 08:57:54.955466 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.954918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.3:39083)
0806 08:57:54.955160 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.955255 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 08:57:54.959710 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.959219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.22.98.2:49807)
0806 08:57:54.959371 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:54.959481 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 08:57:54.971834 32629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:54.972129 32629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:54.977720 32629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b1011f202e44fb3a56b2197c6f8f2b1: Sending DeleteTablet for 3 replicas of tablet 0e9047c03bd24bf0950ed7aa512fe8c6
I20260806 08:57:54.978839 764 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:54 UTC) from {username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:44712
I20260806 08:57:54.979206 571 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:54 UTC) from {username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:48496
I20260806 08:57:54.979637 959 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9047c03bd24bf0950ed7aa512fe8c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:54 UTC) from {username='oryx', principal='oryx/127.22.98.61@KRBTEST.COM'} at 127.0.0.1:36242
I20260806 08:57:54.982514 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 403
I20260806 08:57:54.996086 1245 tablet_replica.cc:330] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: stopping tablet replica
I20260806 08:57:54.996465 1245 raft_consensus.cc:2229] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:54.997146 1244 tablet_replica.cc:330] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: stopping tablet replica
I20260806 08:57:54.997139 1245 raft_consensus.cc:2258] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:54.997375 1244 raft_consensus.cc:2229] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:54.997699 1244 raft_consensus.cc:2258] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:54.998245 1245 ts_tablet_manager.cc:1905] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:54.999118 1244 ts_tablet_manager.cc:1905] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:55.004392 1241 tablet_replica.cc:330] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: stopping tablet replica
I20260806 08:57:55.004587 1241 raft_consensus.cc:2229] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:55.004914 1241 raft_consensus.cc:2258] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:55.006067 1241 ts_tablet_manager.cc:1905] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:55.006814 1245 ts_tablet_manager.cc:1918] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:55.006965 1245 log.cc:1196] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/0e9047c03bd24bf0950ed7aa512fe8c6
I20260806 08:57:55.007433 1245 ts_tablet_manager.cc:1939] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4cf77181c0df4751b893340a4d0f37a8: Deleting consensus metadata
I20260806 08:57:55.008117 1244 ts_tablet_manager.cc:1918] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:55.008267 1244 log.cc:1196] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/0e9047c03bd24bf0950ed7aa512fe8c6
I20260806 08:57:55.008785 1244 ts_tablet_manager.cc:1939] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 4f72a334b34340958f1ba0d3171f79a6: Deleting consensus metadata
I20260806 08:57:55.008762 32606 catalog_manager.cc:4841] TS 4cf77181c0df4751b893340a4d0f37a8 (127.22.98.3:35517): tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (table test-table [id=447efa1804c346beba76ae575b99d525]) successfully deleted
I20260806 08:57:55.009971 32605 catalog_manager.cc:4841] TS 4f72a334b34340958f1ba0d3171f79a6 (127.22.98.1:32931): tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (table test-table [id=447efa1804c346beba76ae575b99d525]) successfully deleted
I20260806 08:57:55.012012 1241 ts_tablet_manager.cc:1918] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:55.012136 1241 log.cc:1196] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/0e9047c03bd24bf0950ed7aa512fe8c6
I20260806 08:57:55.012511 1241 ts_tablet_manager.cc:1939] T 0e9047c03bd24bf0950ed7aa512fe8c6 P 38d1c77a4ec0449dbc1dc6967b7f6ef3: Deleting consensus metadata
I20260806 08:57:55.013412 32606 catalog_manager.cc:4841] TS 38d1c77a4ec0449dbc1dc6967b7f6ef3 (127.22.98.2:39299): tablet 0e9047c03bd24bf0950ed7aa512fe8c6 (table test-table [id=447efa1804c346beba76ae575b99d525]) successfully deleted
I20260806 08:57:55.015887 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:54.984127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.22.98.1:32931)
0806 08:57:54.984315 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:54.984322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:54.996446 (+ 12124us) server_negotiation.cc:407] Connection header received
0806 08:57:54.996664 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:54.996674 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:54.996770 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:54.996893 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:54.999864 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.001033 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.002986 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.003698 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.003828 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:55.008077 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:55.008103 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:55.008126 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:55.008180 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:55.012562 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.013633 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.013643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.013647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.013795 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.014317 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.014324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.014328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.014732 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:55.015002 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:55.015357 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 08:57:55.015522 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 08:57:55.016552 650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:55.132725 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 403
I20260806 08:57:55.156723 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 677
I20260806 08:57:55.174046 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:55.157616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.22.98.2:39299)
0806 08:57:55.157822 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:55.157828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:55.157855 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:57:55.158035 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:55.158041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:55.158127 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:55.158248 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:55.159631 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.160771 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.162139 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.162822 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.162975 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:55.166301 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:55.166325 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:55.166350 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:55.166413 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:55.170683 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.171641 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.171653 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.171656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.171842 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.172241 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.172247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.172250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.172753 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:55.173099 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:55.173403 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 08:57:55.173641 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 08:57:55.174811 850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:55.369656 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 677
I20260806 08:57:55.381716 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:55.380995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33912)
0806 08:57:55.381455 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:55.381547 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":230,"threads_started":1}
I20260806 08:57:55.384595 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:55.384223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40807 (local address 127.0.0.1:33920)
0806 08:57:55.384383 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:55.384459 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 08:57:55.407449 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 874
I20260806 08:57:55.425450 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:55.409225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.22.98.3:35517)
0806 08:57:55.409492 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:55.409498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:55.409518 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:57:55.409598 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:55.409602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:55.409665 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:55.409791 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:55.411332 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.412251 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.413616 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:55.414077 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:55.414219 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:55.417091 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:55.417122 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:55.417147 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:55.417199 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:55.421562 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.422351 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.422362 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.422365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.422556 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:55.422973 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:55.422979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:55.422983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:55.423493 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:55.423875 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:55.424255 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 08:57:55.424512 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260806 08:57:55.426126 1053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:55.563575 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 874
I20260806 08:57:55.590020 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 32441
I20260806 08:57:55.591185 32675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:55.715257 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 32441
I20260806 08:57:55.739434 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 32576
I20260806 08:57:55.740588 455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:57:55.860952 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 32576
2026-08-06T08:57:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11636 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:57:55 dist-test-slave-jpj1 krb5kdc[1259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:57:55 dist-test-slave-jpj1 krb5kdc[1259](info): set up 2 sockets
Aug 06 08:57:55 dist-test-slave-jpj1 krb5kdc[1259](info): commencing operation
krb5kdc: starting...
W20260806 08:57:57.056711 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:57:57 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:57:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab.
Entry for principal oryx/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab.
I20260806 08:57:57.329216 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41723
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab
--principal=oryx/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:57.674455 1275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:57.674767 1275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:57.674906 1275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:57.682399 1275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:57:57.682523 1275 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 08:57:57.682592 1275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:57.682686 1275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:57.682765 1275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:57:57.682853 1275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:57:57.693640 1275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:41723
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.62.keytab
--principal=oryx/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:57.694917 1275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:57.696301 1275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:57.708328 1284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:57.708405 1280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:57.708416 1281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:57.709167 1275 server_base.cc:1034] running on GCE node
I20260806 08:57:57.709954 1275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:57.711360 1275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:57.712575 1275 hybrid_clock.cc:648] HybridClock initialized: now 1786006677712497 us; error 52 us; skew 500 ppm
I20260806 08:57:57.712883 1275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:57 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006677, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:57.718720 1275 init.cc:377] Logged in from keytab as oryx/127.22.98.62@KRBTEST.COM (short username oryx)
I20260806 08:57:57.720295 1275 webserver.cc:460] Webserver started at http://127.22.98.62:43627/ using document root <none> and password file <none>
I20260806 08:57:57.721016 1275 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:57.721120 1275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:57.721469 1275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:57.724643 1275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "a208de19fb1841b9b0faedae07933b73"
format_stamp: "Formatted at 2026-08-06 08:57:57 on dist-test-slave-jpj1"
server_key: "8596f77b343024d246706aa9b6c043ba"
server_key_iv: "0ff427b11082be036c46e52bb6e69537"
server_key_version: "encryptionkey@0"
I20260806 08:57:57.725435 1275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "a208de19fb1841b9b0faedae07933b73"
format_stamp: "Formatted at 2026-08-06 08:57:57 on dist-test-slave-jpj1"
server_key: "8596f77b343024d246706aa9b6c043ba"
server_key_iv: "0ff427b11082be036c46e52bb6e69537"
server_key_version: "encryptionkey@0"
I20260806 08:57:57.730643 1275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 08:57:57.734139 1292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:57.735457 1275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:57.735664 1275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "a208de19fb1841b9b0faedae07933b73"
format_stamp: "Formatted at 2026-08-06 08:57:57 on dist-test-slave-jpj1"
server_key: "8596f77b343024d246706aa9b6c043ba"
server_key_iv: "0ff427b11082be036c46e52bb6e69537"
server_key_version: "encryptionkey@0"
I20260806 08:57:57.735875 1275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:57.768841 1275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:57.773088 1275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:57.773370 1275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:57.794895 1275 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:41723
I20260806 08:57:57.794919 1375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:41723 every 8 connection(s)
I20260806 08:57:57.796427 1275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:57:57.800381 1377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:57.805862 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 1275
I20260806 08:57:57.806083 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:57:57.806915 22920 external_mini_cluster.cc:1428] Setting key afbcdd511e1a0ef86c5a40839cea6990
I20260806 08:57:57.812300 1377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Bootstrap starting.
I20260806 08:57:57.816061 1377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:57.817369 1377 log.cc:824] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:57.821106 1377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: No bootstrap required, opened a new log
Aug 06 08:57:57 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:57.830605 1377 raft_consensus.cc:348] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } }
I20260806 08:57:57.831004 1377 raft_consensus.cc:374] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:57.831108 1377 raft_consensus.cc:729] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a208de19fb1841b9b0faedae07933b73, State: Initialized, Role: FOLLOWER
I20260806 08:57:57.831585 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:57.809200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39158 (local address 127.22.98.62:41723)
0806 08:57:57.810040 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:57.810052 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:57.810107 (+ 55us) server_negotiation.cc:407] Connection header received
0806 08:57:57.811892 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:57.811923 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:57.812354 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:57.812731 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:57.814570 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:57.815981 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:57.817707 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:57.818833 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:57.818983 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:57.823996 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:57.824008 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:57.824029 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:57.824077 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:57.827273 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:57.828133 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:57.828141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:57.828148 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:57.828352 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:57.828759 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:57.828762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:57.828764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:57.829357 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:57.829634 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:57.830034 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 08:57:57.830400 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":403,"threads_started":1}
I20260806 08:57:57.831840 1377 consensus_queue.cc:260] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } }
I20260806 08:57:57.832093 1377 raft_consensus.cc:388] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:57:57.832247 1377 raft_consensus.cc:482] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:57:57.832422 1377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:57.834767 1377 raft_consensus.cc:504] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } }
I20260806 08:57:57.835291 1377 leader_election.cc:302] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a208de19fb1841b9b0faedae07933b73; no voters:
I20260806 08:57:57.835916 1377 leader_election.cc:288] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:57:57.836048 1384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:57.836818 1384 raft_consensus.cc:686] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [term 1 LEADER]: Becoming Leader. State: Replica: a208de19fb1841b9b0faedae07933b73, State: Running, Role: LEADER
I20260806 08:57:57.837369 1384 consensus_queue.cc:237] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } }
I20260806 08:57:57.837972 1377 sys_catalog.cc:564] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:57:57.840543 1385 sys_catalog.cc:455] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a208de19fb1841b9b0faedae07933b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } } }
I20260806 08:57:57.840945 1385 sys_catalog.cc:458] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:57.851151 1388 sys_catalog.cc:455] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a208de19fb1841b9b0faedae07933b73. Latest consensus state: current_term: 1 leader_uuid: "a208de19fb1841b9b0faedae07933b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a208de19fb1841b9b0faedae07933b73" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41723 } } }
I20260806 08:57:57.851454 1388 sys_catalog.cc:458] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73 [sys.catalog]: This master's current role is: LEADER
I20260806 08:57:57.855003 1397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:57:57.860703 1397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:57:57.870414 1397 catalog_manager.cc:1347] Generated new cluster ID: 9a6497ea606c46cfbb1c2115243b11a8
I20260806 08:57:57.870544 1397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:57:57.900377 1397 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:57:57.902302 1397 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:57:57.910190 1397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Generated new TSK 0
I20260806 08:57:57.911005 1397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
I20260806 08:57:57.980414 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41723
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:58.331722 1413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:58.332015 1413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:58.332162 1413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:58.339526 1413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:58.339691 1413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:58.339829 1413 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:58.339944 1413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:57:58.350109 1413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:58.351459 1413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:58.352880 1413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:58.365825 1419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:58.365782 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:58.365748 1424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:58.372298 1413 server_base.cc:1034] running on GCE node
I20260806 08:57:58.372946 1413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:58.373948 1413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:58.375202 1413 hybrid_clock.cc:648] HybridClock initialized: now 1786006678375097 us; error 63 us; skew 500 ppm
I20260806 08:57:58.375499 1413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:58 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006678, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:58.378784 1413 init.cc:377] Logged in from keytab as oryx/127.22.98.1@KRBTEST.COM (short username oryx)
I20260806 08:57:58.380450 1413 webserver.cc:460] Webserver started at http://127.22.98.1:46173/ using document root <none> and password file <none>
I20260806 08:57:58.381196 1413 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:58.381291 1413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:58.381628 1413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:58.384691 1413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "66adedc5560f4d43bec6a4a9c1323977"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "125743304309c0661d7546093be231e9"
server_key_iv: "390077cf9e25fb57798a3f8cb0d46aff"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.385506 1413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "66adedc5560f4d43bec6a4a9c1323977"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "125743304309c0661d7546093be231e9"
server_key_iv: "390077cf9e25fb57798a3f8cb0d46aff"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.390837 1413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 08:57:58.394421 1432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:58.395807 1413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:57:58.395960 1413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "66adedc5560f4d43bec6a4a9c1323977"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "125743304309c0661d7546093be231e9"
server_key_iv: "390077cf9e25fb57798a3f8cb0d46aff"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.396206 1413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:58.410748 1413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:58.415010 1413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:58.415279 1413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:58.418011 1413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:58.418107 1413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:58.418267 1413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:58.418337 1413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:58 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006678, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:58.443840 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.418615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:58435 (local address 127.22.98.62:41723)
0806 08:57:58.419235 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:58.419242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:58.420336 (+ 1094us) server_negotiation.cc:407] Connection header received
0806 08:57:58.422423 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:58.422429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:58.422515 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:58.422615 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:58.424904 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.425900 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.428387 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.428941 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.429089 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:58.434539 (+ 5450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:58.434547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:58.434552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:58.434611 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:58.438055 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:58.439371 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:58.439377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:58.439381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:58.439587 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:58.440213 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:58.440219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:58.440223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:58.440658 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:58.440818 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:58.443126 (+ 2308us) server_negotiation.cc:299] Negotiation successful
0806 08:57:58.443392 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":375,"threads_started":1}
I20260806 08:57:58.445171 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.419259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.1:58435)
0806 08:57:58.420162 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:58.420211 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:58.422142 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:58.422902 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:58.422915 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:58.423778 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:58.424694 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.424727 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.426079 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.426089 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.428245 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.428258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.429282 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.429297 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.429710 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:58.430682 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:58.430719 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:58.434212 (+ 3493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:58.438272 (+ 4060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:58.438288 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:58.438316 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:58.439211 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:58.439713 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:58.439717 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:58.439721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:58.440061 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:58.440802 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:58.440812 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:58.441416 (+ 604us) client_negotiation.cc:769] Sending connection context
0806 08:57:58.441810 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 08:57:58.442525 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":377,"threads_started":1}
I20260806 08:57:58.454262 1413 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:43263
I20260806 08:57:58.454319 1591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:43263 every 8 connection(s)
I20260806 08:57:58.455698 1413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:57:58.457672 1592 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41723
I20260806 08:57:58.457887 1592 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:58.458654 1592 heartbeater.cc:511] Master 127.22.98.62:41723 requested a full tablet report, sending...
I20260806 08:57:58.459100 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 1413
I20260806 08:57:58.459304 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:57:58.460165 22920 external_mini_cluster.cc:1428] Setting key 387d691a6923ea4c375f6c2311c81bc3
I20260806 08:57:58.461628 1315 ts_manager.cc:194] Registered new tserver with Master: 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263)
I20260806 08:57:58.463981 1315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:58435
I20260806 08:57:58.467028 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.454069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:44015 (local address 127.22.98.62:41723)
0806 08:57:58.454243 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:58.454249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:58.454438 (+ 189us) server_negotiation.cc:407] Connection header received
0806 08:57:58.454693 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:58.454697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:58.454757 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:58.454836 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:58.455835 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.456817 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.458367 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.459002 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.459153 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:58.461271 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:58.461277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:58.461280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:58.461316 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:58.464106 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:58.464836 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:58.464842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:58.464846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:58.465017 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:58.465497 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:58.465502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:58.465505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:58.465870 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:58.466040 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:58.466474 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 08:57:58.466698 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 08:57:58.467064 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.453509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.1:44015)
0806 08:57:58.454314 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:58.454338 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:58.454561 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:58.454877 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:58.454882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:58.455346 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:58.455670 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.455684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.456936 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.456941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.458224 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.458237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.459396 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.459403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.459608 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:58.460196 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:58.460212 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:58.461073 (+ 861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:58.464297 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:58.464303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:58.464308 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:58.464708 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:58.465150 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:58.465155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:58.465159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:58.465372 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:58.465999 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:58.466004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:58.466198 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 08:57:58.466336 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 08:57:58.466632 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":732,"threads_started":1}
WARNING: no policy specified for oryx/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
I20260806 08:57:58.514719 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41723
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:58.885252 1596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:58.885547 1596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:58.885645 1596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:58.892962 1596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:58.893078 1596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:58.893170 1596 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:58.893254 1596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:57:58.903415 1596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:58.904729 1596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:58.906152 1596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:58.918962 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:58.919003 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:58.919030 1602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:58.920064 1596 server_base.cc:1034] running on GCE node
I20260806 08:57:58.920709 1596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:58.921576 1596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:58.922786 1596 hybrid_clock.cc:648] HybridClock initialized: now 1786006678922724 us; error 42 us; skew 500 ppm
I20260806 08:57:58.923079 1596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:58 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006678, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:58.926327 1596 init.cc:377] Logged in from keytab as oryx/127.22.98.2@KRBTEST.COM (short username oryx)
I20260806 08:57:58.927842 1596 webserver.cc:460] Webserver started at http://127.22.98.2:46373/ using document root <none> and password file <none>
I20260806 08:57:58.928527 1596 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:58.928617 1596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:58.928905 1596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:58.931954 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "679d49b23e374f5fb076cba3735e6963"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "3808becd5bdf745b872b44e173b1dbb3"
server_key_iv: "1dd0d60a9bf3932c4c35e27656dce75f"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.932709 1596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "679d49b23e374f5fb076cba3735e6963"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "3808becd5bdf745b872b44e173b1dbb3"
server_key_iv: "1dd0d60a9bf3932c4c35e27656dce75f"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.937829 1596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:57:58.941537 1611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:58.942993 1596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 08:57:58.943146 1596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "679d49b23e374f5fb076cba3735e6963"
format_stamp: "Formatted at 2026-08-06 08:57:58 on dist-test-slave-jpj1"
server_key: "3808becd5bdf745b872b44e173b1dbb3"
server_key_iv: "1dd0d60a9bf3932c4c35e27656dce75f"
server_key_version: "encryptionkey@0"
I20260806 08:57:58.943361 1596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:58.975797 1596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:58.980118 1596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:58.980386 1596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:58.983153 1596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:58.983238 1596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:58.983409 1596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:58.983485 1596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:58 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006678, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:59.007141 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.984158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:53325 (local address 127.22.98.62:41723)
0806 08:57:58.984583 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:58.984588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:58.985933 (+ 1345us) server_negotiation.cc:407] Connection header received
0806 08:57:58.988640 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:58.988646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:58.988746 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:58.988871 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:58.991315 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.992386 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.995006 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.995500 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.995659 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.000540 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:59.000548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:59.000553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:59.000598 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:59.004016 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.004758 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.004765 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.004769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.004907 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.005346 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.005352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.005356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.005637 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:59.005783 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.006512 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.006744 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":214,"threads_started":1}
I20260806 08:57:59.008509 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:58.984843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.2:53325)
0806 08:57:58.985735 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:58.985812 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:58.988142 (+ 2330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:58.989191 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:58.989206 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:58.990056 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:58.991058 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.991092 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.992604 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.992613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.994791 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:58.994804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:58.995785 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:58.995798 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:58.996071 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:58.996863 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:58.996897 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:59.000299 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:59.004159 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.004169 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.004187 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.004636 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.005025 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.005029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.005031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.005235 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.005785 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:59.005794 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:59.006145 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.006399 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.006968 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":400,"threads_started":1}
I20260806 08:57:59.029031 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.015029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.2:44381)
0806 08:57:59.015934 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.015951 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.016085 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.016393 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.016397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.016852 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:59.017143 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.017152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.018119 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.018125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.019137 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.019149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.020021 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.020028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.020196 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.020847 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:59.020877 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:59.022044 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:59.025759 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.025766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.025770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.026473 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.026922 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.026928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.026932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.027115 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.027975 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:59.027980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:59.028174 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.028311 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.028680 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":603,"threads_started":1}
I20260806 08:57:59.029078 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.014619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:44381 (local address 127.22.98.62:41723)
0806 08:57:59.014799 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.014806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.016033 (+ 1227us) server_negotiation.cc:407] Connection header received
0806 08:57:59.016193 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.016198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.016269 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.016352 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:59.017284 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.017993 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.019274 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.019835 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.019957 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.022264 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:59.022270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:59.022273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:59.022304 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:59.025512 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.026623 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.026628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.026632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.026790 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.027269 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.027274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.027277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.027822 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:59.027979 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.028448 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.028698 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 08:57:59.040783 1596 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:36263
I20260806 08:57:59.040814 1765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:36263 every 8 connection(s)
I20260806 08:57:59.042060 1596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:57:59.044486 1767 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41723
I20260806 08:57:59.044695 1767 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:59.045331 1767 heartbeater.cc:511] Master 127.22.98.62:41723 requested a full tablet report, sending...
I20260806 08:57:59.046670 1315 ts_manager.cc:194] Registered new tserver with Master: 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:57:59.047582 1315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:53325
I20260806 08:57:59.051923 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 1596
I20260806 08:57:59.052063 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:57:59.052477 22920 external_mini_cluster.cc:1428] Setting key 122294e771f55e71ad016ecb599bf199
WARNING: no policy specified for oryx/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
I20260806 08:57:59.110066 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41723
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:57:59.454918 1772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:57:59.455207 1772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:57:59.455340 1772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:57:59.462767 1772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:57:59.462893 1772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:57:59.463021 1772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 08:57:59.463114 1772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:57:59.467835 1592 heartbeater.cc:503] Master 127.22.98.62:41723 was elected leader, sending a full tablet report...
I20260806 08:57:59.473328 1772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.98.62:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:57:59.474656 1772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:57:59.476107 1772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:57:59.489439 1780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:59.489393 1783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:57:59.489836 1779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:57:59.490293 1772 server_base.cc:1034] running on GCE node
I20260806 08:57:59.490913 1772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:57:59.491878 1772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:57:59.493108 1772 hybrid_clock.cc:648] HybridClock initialized: now 1786006679493045 us; error 38 us; skew 500 ppm
I20260806 08:57:59.493397 1772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006679, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:57:59.496502 1772 init.cc:377] Logged in from keytab as oryx/127.22.98.3@KRBTEST.COM (short username oryx)
I20260806 08:57:59.498020 1772 webserver.cc:460] Webserver started at http://127.22.98.3:39581/ using document root <none> and password file <none>
I20260806 08:57:59.498725 1772 fs_manager.cc:359] Metadata directory not provided
I20260806 08:57:59.498842 1772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:57:59.499184 1772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:57:59.502370 1772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "9abe67e53e0b4eb7adb40bd8b700268a"
format_stamp: "Formatted at 2026-08-06 08:57:59 on dist-test-slave-jpj1"
server_key: "7a513a5c25ccfa017f59891608ed0fb0"
server_key_iv: "e4a90af3806a2e52e1204e960e29f64e"
server_key_version: "encryptionkey@0"
I20260806 08:57:59.503139 1772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "9abe67e53e0b4eb7adb40bd8b700268a"
format_stamp: "Formatted at 2026-08-06 08:57:59 on dist-test-slave-jpj1"
server_key: "7a513a5c25ccfa017f59891608ed0fb0"
server_key_iv: "e4a90af3806a2e52e1204e960e29f64e"
server_key_version: "encryptionkey@0"
I20260806 08:57:59.508368 1772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 08:57:59.511878 1790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:59.513460 1772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:57:59.513631 1772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "9abe67e53e0b4eb7adb40bd8b700268a"
format_stamp: "Formatted at 2026-08-06 08:57:59 on dist-test-slave-jpj1"
server_key: "7a513a5c25ccfa017f59891608ed0fb0"
server_key_iv: "e4a90af3806a2e52e1204e960e29f64e"
server_key_version: "encryptionkey@0"
I20260806 08:57:59.513891 1772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:57:59.529636 1772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:57:59.534066 1772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:57:59.534325 1772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:57:59.537191 1772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:57:59.537276 1772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:57:59.537421 1772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:57:59.537497 1772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006679, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:59.564267 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.537707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:47305 (local address 127.22.98.62:41723)
0806 08:57:59.538493 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.538499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.539837 (+ 1338us) server_negotiation.cc:407] Connection header received
0806 08:57:59.542829 (+ 2992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.542835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.542938 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.543060 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:59.545592 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.546614 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.548946 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.549376 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.549497 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.554734 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:59.554743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:59.554748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:59.554793 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:59.558231 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.559445 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.559452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.559456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.559669 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.560410 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.560415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.560418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.560805 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:59.560973 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.563777 (+ 2804us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.563967 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":538,"threads_started":1}
I20260806 08:57:59.564654 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.538365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.3:47305)
0806 08:57:59.539482 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.539572 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.542310 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.543437 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.543451 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.544340 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:59.545324 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.545355 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.546809 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.546813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.548722 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.548743 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.549655 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.549664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.549992 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.550853 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:59.550904 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:59.554368 (+ 3464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:59.558436 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.558447 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.558474 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.559232 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.559826 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.559831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.559835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.560221 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.561063 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:59.561076 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:59.561675 (+ 599us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.562065 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.562801 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":387,"threads_started":1}
I20260806 08:57:59.584089 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.570404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:50743 (local address 127.22.98.62:41723)
0806 08:57:59.570578 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.570584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.571374 (+ 790us) server_negotiation.cc:407] Connection header received
0806 08:57:59.571506 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.571512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.571635 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.571746 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:59.573071 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.574177 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.575874 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.576329 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.576445 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.578210 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:59.578215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:59.578219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:59.578249 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:59.581206 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.582047 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.582053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.582057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.582198 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.582557 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.582563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.582567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.582931 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:59.583071 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.583484 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.583734 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 08:57:59.584233 1907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.570636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.22.98.3:50743)
0806 08:57:59.571268 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.571287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.571401 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.571806 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.571810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.572452 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:57:59.572891 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.572907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.574313 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.574317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.575737 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.575751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.576515 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.576522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.576683 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.577312 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:57:59.577329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:57:59.578019 (+ 690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:57:59.581331 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.581337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.581341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.581931 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.582308 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:57:59.582317 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:57:59.582321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:57:59.582455 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:57:59.583038 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:57:59.583042 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:57:59.583241 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.583361 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.583741 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":337,"threads_started":1}
I20260806 08:57:59.587749 1772 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:45463
I20260806 08:57:59.587749 1948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:45463 every 8 connection(s)
I20260806 08:57:59.589010 1772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:57:59.590982 1949 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41723
I20260806 08:57:59.591215 1949 heartbeater.cc:464] Registering TS with master...
I20260806 08:57:59.591890 1949 heartbeater.cc:511] Master 127.22.98.62:41723 requested a full tablet report, sending...
I20260806 08:57:59.593335 1315 ts_manager.cc:194] Registered new tserver with Master: 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463)
I20260806 08:57:59.594127 1315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:47305
I20260806 08:57:59.597087 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 1772
I20260806 08:57:59.597227 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:57:59.597612 22920 external_mini_cluster.cc:1428] Setting key 507b10760fe6d02b5573a33c22c7259a
I20260806 08:57:59.600800 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 08:57:59.660012 1806 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:57:59.649069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.22.98.62:41723)
0806 08:57:59.649300 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.649308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.649649 (+ 341us) server_negotiation.cc:407] Connection header received
0806 08:57:59.649855 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.649861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.649944 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.650036 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:59.651481 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.652256 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.653513 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.653935 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.654051 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.659677 (+ 5626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39164: BlockingRecv error: recv got EOF from 127.0.0.1:39164 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 08:57:59 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:57:59.677228 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.663692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.22.98.62:41723)
0806 08:57:59.663845 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.663849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.664252 (+ 403us) server_negotiation.cc:407] Connection header received
0806 08:57:59.664414 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.664419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.664483 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.664568 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:57:59.665724 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.666491 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.667898 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.668338 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.668431 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.672086 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:57:59.672092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:57:59.672095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:57:59.672130 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:57:59.674608 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.675274 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.675278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.675281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.675418 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:57:59.675850 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:57:59.675854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:57:59.675856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:57:59.676136 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:57:59.676395 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.676690 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.676919 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 08:57:59.681126 1315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:59.688057 1315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:59.716441 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.703102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.22.98.1:43263)
0806 08:57:59.705091 (+ 1989us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.705102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.708129 (+ 3027us) server_negotiation.cc:407] Connection header received
0806 08:57:59.708582 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.708591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.708865 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.709032 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:59.709769 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.711222 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.713885 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.715169 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.715314 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.715529 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.715847 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.716091 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1444,"thread_start_us":1303,"threads_started":1}
I20260806 08:57:59.717235 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.703125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:36263 (local address 127.0.0.1:55760)
0806 08:57:59.708368 (+ 5243us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.708384 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.708489 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.709832 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.709836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.709860 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:59.710275 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.710289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.712020 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.712025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.714306 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.714322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.716398 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.716404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.716577 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.716610 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.716704 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.716822 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5013,"mutex_wait_us":27,"thread_start_us":4864,"threads_started":1}
I20260806 08:57:59.717561 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.703625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.22.98.2:36263)
0806 08:57:59.709342 (+ 5717us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.709349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.709373 (+ 24us) server_negotiation.cc:407] Connection header received
0806 08:57:59.709463 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.709468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.709645 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.709828 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:59.710472 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.711851 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.714477 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.715915 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.716043 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.716246 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.717064 (+ 818us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.717244 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5580,"thread_start_us":5435,"threads_started":1}
I20260806 08:57:59.717612 1974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.701952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43263 (local address 127.0.0.1:57958)
0806 08:57:59.707948 (+ 5996us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.708000 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.708306 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.709309 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.709315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.709339 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:59.709620 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.709629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.711353 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.711358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.713727 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.713741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.715366 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.715372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.715586 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.715645 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.715785 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.717167 (+ 1382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5644,"mutex_wait_us":29,"thread_start_us":1633,"threads_started":1}
I20260806 08:57:59.718097 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.702630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.22.98.3:45463)
0806 08:57:59.705699 (+ 3069us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.705707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.709889 (+ 4182us) server_negotiation.cc:407] Connection header received
0806 08:57:59.710213 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.710221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.710469 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.710633 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:59.711230 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.712857 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.715179 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.716597 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.716717 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.716955 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.717601 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.717779 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2880,"thread_start_us":2679,"threads_started":1}
I20260806 08:57:59.718936 1975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.702233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45463 (local address 127.0.0.1:34308)
0806 08:57:59.709774 (+ 7541us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.709791 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.710005 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.710634 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.710638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.710664 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:59.711034 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.711044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.713015 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.713020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.715014 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.715030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.717116 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.717124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.717373 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.717407 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.718446 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.718580 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5426,"mutex_wait_us":4,"thread_start_us":3638,"threads_started":1}
I20260806 08:57:59.721422 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa (DEFAULT_TABLE table=test-table [id=afdde2c27f9647888f7776e7668eb73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.721637 1494 tablet_service.cc:1467] Processing CreateTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa (DEFAULT_TABLE table=test-table [id=afdde2c27f9647888f7776e7668eb73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.722669 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e46d3432664e60b0f6a32124fe6dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.722297 1862 tablet_service.cc:1467] Processing CreateTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa (DEFAULT_TABLE table=test-table [id=afdde2c27f9647888f7776e7668eb73e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.723059 1494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e46d3432664e60b0f6a32124fe6dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.723675 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e46d3432664e60b0f6a32124fe6dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.734127 1981 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Bootstrap starting.
I20260806 08:57:59.737910 1982 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Bootstrap starting.
I20260806 08:57:59.738438 1981 tablet_bootstrap.cc:654] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.739322 1983 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Bootstrap starting.
I20260806 08:57:59.740181 1981 log.cc:824] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:59.741636 1982 tablet_bootstrap.cc:654] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.742475 1983 tablet_bootstrap.cc:654] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.743645 1981 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: No bootstrap required, opened a new log
I20260806 08:57:59.743638 1983 log.cc:824] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:59.743183 1982 log.cc:824] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Log is configured to *not* fsync() on all Append() calls
I20260806 08:57:59.744000 1981 ts_tablet_manager.cc:1397] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 08:57:59.749322 1983 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: No bootstrap required, opened a new log
I20260806 08:57:59.749611 1983 ts_tablet_manager.cc:1397] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 08:57:59.750185 1982 tablet_bootstrap.cc:492] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: No bootstrap required, opened a new log
I20260806 08:57:59.750473 1982 ts_tablet_manager.cc:1397] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260806 08:57:59.754890 1981 raft_consensus.cc:348] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.755403 1981 raft_consensus.cc:374] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.755519 1981 raft_consensus.cc:729] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679d49b23e374f5fb076cba3735e6963, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.756275 1981 consensus_queue.cc:260] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.757529 1767 heartbeater.cc:503] Master 127.22.98.62:41723 was elected leader, sending a full tablet report...
I20260806 08:57:59.757663 1983 raft_consensus.cc:348] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.758037 1983 raft_consensus.cc:374] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.758141 1983 raft_consensus.cc:729] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.758755 1983 consensus_queue.cc:260] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.759729 1981 ts_tablet_manager.cc:1428] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260806 08:57:59.762015 1982 raft_consensus.cc:348] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.762439 1982 raft_consensus.cc:374] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.762553 1982 raft_consensus.cc:729] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.763278 1982 consensus_queue.cc:260] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.765667 1982 ts_tablet_manager.cc:1428] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260806 08:57:59.766604 1949 heartbeater.cc:503] Master 127.22.98.62:41723 was elected leader, sending a full tablet report...
I20260806 08:57:59.768021 1983 ts_tablet_manager.cc:1428] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent starting tablet: real 0.018s user 0.009s sys 0.009s
I20260806 08:57:59.770990 1992 raft_consensus.cc:482] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:57:59.771286 1992 raft_consensus.cc:504] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.772900 1992 leader_election.cc:288] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:57:59.779277 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.773231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:36263 (local address 127.22.98.3:49861)
0806 08:57:59.773537 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.773562 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.773692 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.774089 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.774092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.774115 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:59.774330 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.774343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.775557 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.775564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.777307 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.777320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.778508 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.778514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.778694 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.778720 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.778803 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.778973 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 08:57:59.779263 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.773415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:49861 (local address 127.22.98.2:36263)
0806 08:57:59.773626 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.773632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.773654 (+ 22us) server_negotiation.cc:407] Connection header received
0806 08:57:59.773876 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.773881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.773962 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.774062 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:59.774474 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.775407 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.777459 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.778347 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.778474 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.778581 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.778843 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.778979 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 08:57:59.779985 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.773464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:34421 (local address 127.22.98.1:43263)
0806 08:57:59.773648 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.773653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.774285 (+ 632us) server_negotiation.cc:407] Connection header received
0806 08:57:59.774422 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.774427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.774505 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.774569 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:57:59.775151 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.776068 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.778047 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.778859 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.778997 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.779093 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.779607 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.779761 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 08:57:59.779976 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.773285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43263 (local address 127.22.98.3:34421)
0806 08:57:59.774168 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 08:57:59.774192 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:57:59.774312 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:57:59.774609 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:57:59.774613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:57:59.774638 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:57:59.775003 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.775015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.776201 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.776205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.777889 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:57:59.777903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.779261 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.779269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:57:59.779418 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.779455 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 08:57:59.779583 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 08:57:59.779704 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":528,"threads_started":1}
I20260806 08:57:59.780427 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e46d3432664e60b0f6a32124fe6dfa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963" is_pre_election: true
I20260806 08:57:59.781024 1707 raft_consensus.cc:2454] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:57:59.781044 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e46d3432664e60b0f6a32124fe6dfa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977" is_pre_election: true
I20260806 08:57:59.781595 1524 raft_consensus.cc:2454] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:57:59.781813 1799 leader_election.cc:302] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 679d49b23e374f5fb076cba3735e6963, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:57:59.782214 1992 raft_consensus.cc:2781] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:57:59.782347 1992 raft_consensus.cc:482] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:57:59.782446 1992 raft_consensus.cc:3037] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:59.784801 1992 raft_consensus.cc:504] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
I20260806 08:57:59.785418 1992 leader_election.cc:288] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Requested vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:57:59.785972 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e46d3432664e60b0f6a32124fe6dfa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977"
I20260806 08:57:59.786005 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e46d3432664e60b0f6a32124fe6dfa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963"
I20260806 08:57:59.786206 1707 raft_consensus.cc:3037] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:59.786206 1524 raft_consensus.cc:3037] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:57:59.789464 1707 raft_consensus.cc:2454] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:57:59.789464 1524 raft_consensus.cc:2454] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:57:59.789963 1793 leader_election.cc:302] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:57:59.790287 1992 raft_consensus.cc:2781] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:57:59.790761 1992 raft_consensus.cc:686] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Becoming Leader. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Running, Role: LEADER
I20260806 08:57:59.791230 1992 consensus_queue.cc:237] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } }
W20260806 08:57:59.795689 1768 tablet.cc:2369] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:59.796671 1315 catalog_manager.cc:5495] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3). New cstate: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260806 08:57:59.825608 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:57:59.820389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34324 (local address 127.22.98.3:45463)
0806 08:57:59.820561 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 08:57:59.820566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:57:59.820584 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:57:59.820824 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:57:59.820829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:57:59.820921 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:57:59.821058 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:57:59.821386 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.822181 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.823528 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:57:59.824011 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:57:59.824126 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:57:59.824509 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:57:59.825145 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:57:59.825275 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 08:57:59.825315 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 08:57:59.825386 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 08:57:59.834481 1524 raft_consensus.cc:1261] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:59.834491 1707 raft_consensus.cc:1261] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:57:59.835599 1995 consensus_queue.cc:1035] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:57:59.835975 1992 consensus_queue.cc:1035] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 08:57:59.840560 1950 tablet.cc:2369] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:57:59.862509 1315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:59.862779 1315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:57:59.866215 1315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Sending DeleteTablet for 3 replicas of tablet 03e46d3432664e60b0f6a32124fe6dfa
I20260806 08:57:59.867370 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:59 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34308
I20260806 08:57:59.868052 1494 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:59 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57958
I20260806 08:57:59.868165 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e46d3432664e60b0f6a32124fe6dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:57:59 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55760
I20260806 08:57:59.868515 1315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:57:59.869254 1315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:57:59.872265 2013 tablet_replica.cc:330] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: stopping tablet replica
I20260806 08:57:59.874255 2013 raft_consensus.cc:2229] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:59.874855 2013 raft_consensus.cc:2258] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:59.877157 1861 tablet_service.cc:1467] Processing CreateTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 (DEFAULT_TABLE table=test-table [id=de7f94535a504b6f85af8dc6621feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.877449 1678 tablet_service.cc:1467] Processing CreateTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 (DEFAULT_TABLE table=test-table [id=de7f94535a504b6f85af8dc6621feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.877781 1861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63819ad3bc4acd9f939fcaaad62d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.878046 1678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63819ad3bc4acd9f939fcaaad62d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.878136 2013 ts_tablet_manager.cc:1905] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:59.878739 1494 tablet_service.cc:1467] Processing CreateTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 (DEFAULT_TABLE table=test-table [id=de7f94535a504b6f85af8dc6621feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:57:59.880214 1494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db63819ad3bc4acd9f939fcaaad62d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:57:59.884285 1983 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Bootstrap starting.
I20260806 08:57:59.885104 2013 ts_tablet_manager.cc:1918] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:59.885264 2013 log.cc:1196] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/03e46d3432664e60b0f6a32124fe6dfa
I20260806 08:57:59.885864 2013 ts_tablet_manager.cc:1939] T 03e46d3432664e60b0f6a32124fe6dfa P 66adedc5560f4d43bec6a4a9c1323977: Deleting consensus metadata
I20260806 08:57:59.886049 1983 tablet_bootstrap.cc:654] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.887209 1981 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Bootstrap starting.
I20260806 08:57:59.887733 1293 catalog_manager.cc:4841] TS 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263): tablet 03e46d3432664e60b0f6a32124fe6dfa (table test-table [id=afdde2c27f9647888f7776e7668eb73e]) successfully deleted
I20260806 08:57:59.888981 1981 tablet_bootstrap.cc:654] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.890303 1982 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Bootstrap starting.
I20260806 08:57:59.890676 2014 tablet_replica.cc:330] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: stopping tablet replica
I20260806 08:57:59.891160 2014 raft_consensus.cc:2229] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:57:59.891728 2014 raft_consensus.cc:2258] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:59.892560 2012 tablet_replica.cc:330] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: stopping tablet replica
I20260806 08:57:59.892993 2012 raft_consensus.cc:2229] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:57:59.893272 1982 tablet_bootstrap.cc:654] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Neither blocks nor log segments found. Creating new log.
I20260806 08:57:59.893321 1981 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: No bootstrap required, opened a new log
I20260806 08:57:59.893424 1981 ts_tablet_manager.cc:1397] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 08:57:59.893700 2012 raft_consensus.cc:2258] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:57:59.893981 1981 raft_consensus.cc:348] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.894202 1981 raft_consensus.cc:374] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.894296 1981 raft_consensus.cc:729] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679d49b23e374f5fb076cba3735e6963, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.894541 1981 consensus_queue.cc:260] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.895053 1981 ts_tablet_manager.cc:1428] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 08:57:59.896000 2012 ts_tablet_manager.cc:1905] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:59.902506 2014 ts_tablet_manager.cc:1905] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:57:59.904912 1982 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: No bootstrap required, opened a new log
I20260806 08:57:59.905144 1982 ts_tablet_manager.cc:1397] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260806 08:57:59.905207 1983 tablet_bootstrap.cc:492] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: No bootstrap required, opened a new log
I20260806 08:57:59.905365 1983 ts_tablet_manager.cc:1397] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.006s
I20260806 08:57:59.906073 1982 raft_consensus.cc:348] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.906325 1982 raft_consensus.cc:374] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.906304 1983 raft_consensus.cc:348] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.906436 1982 raft_consensus.cc:729] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.906523 1983 raft_consensus.cc:374] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:57:59.906632 1983 raft_consensus.cc:729] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Initialized, Role: FOLLOWER
I20260806 08:57:59.906765 1982 consensus_queue.cc:260] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.906913 1983 consensus_queue.cc:260] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:57:59.907510 1983 ts_tablet_manager.cc:1428] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 08:57:59.908370 1982 ts_tablet_manager.cc:1428] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 08:57:59.909264 2014 ts_tablet_manager.cc:1918] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:59.909446 2014 log.cc:1196] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/03e46d3432664e60b0f6a32124fe6dfa
I20260806 08:57:59.910920 2014 ts_tablet_manager.cc:1939] T 03e46d3432664e60b0f6a32124fe6dfa P 679d49b23e374f5fb076cba3735e6963: Deleting consensus metadata
I20260806 08:57:59.912662 1297 catalog_manager.cc:4841] TS 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263): tablet 03e46d3432664e60b0f6a32124fe6dfa (table test-table [id=afdde2c27f9647888f7776e7668eb73e]) successfully deleted
I20260806 08:57:59.906556 2012 ts_tablet_manager.cc:1918] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:57:59.915732 2012 log.cc:1196] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/03e46d3432664e60b0f6a32124fe6dfa
I20260806 08:57:59.916182 2012 ts_tablet_manager.cc:1939] T 03e46d3432664e60b0f6a32124fe6dfa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting consensus metadata
I20260806 08:57:59.917693 1295 catalog_manager.cc:4841] TS 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463): tablet 03e46d3432664e60b0f6a32124fe6dfa (table test-table [id=afdde2c27f9647888f7776e7668eb73e]) successfully deleted
W20260806 08:57:59.958727 1593 tablet.cc:2369] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:58:00.104416 1993 raft_consensus.cc:482] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:58:00.104658 1993 raft_consensus.cc:504] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.106417 1993 leader_election.cc:288] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263), 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463)
I20260806 08:58:00.114790 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.107062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:51407 (local address 127.22.98.3:45463)
0806 08:58:00.107591 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.107598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.108370 (+ 772us) server_negotiation.cc:407] Connection header received
0806 08:58:00.108550 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.108555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.108632 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.108727 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:00.109385 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.110549 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.112938 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.113788 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.113915 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.114024 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.114348 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.114483 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
I20260806 08:58:00.114833 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.106793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45463 (local address 127.22.98.1:51407)
0806 08:58:00.108257 (+ 1464us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.108275 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.108412 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.108752 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.108756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.108785 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:00.109188 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.109201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.110718 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.110724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.112775 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.112786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.114009 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.114015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.114177 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.114201 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.114293 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.114428 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1331,"mutex_wait_us":67,"thread_start_us":802,"threads_started":1}
I20260806 08:58:00.115103 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.107166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:51095 (local address 127.22.98.2:36263)
0806 08:58:00.107373 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.107380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.108048 (+ 668us) server_negotiation.cc:407] Connection header received
0806 08:58:00.108344 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.108350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.108431 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.108513 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:00.109256 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.110129 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.112684 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.113852 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.114003 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.114102 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.114682 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.114816 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 08:58:00.115921 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.106750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:36263 (local address 127.22.98.1:51095)
0806 08:58:00.107919 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.107941 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.108177 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.108573 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.108578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.108602 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:00.109045 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.109062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.110267 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.110271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.112512 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.112528 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.114304 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.114311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.114484 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.114518 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.115395 (+ 877us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.115568 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":49,"thread_start_us":1189,"threads_started":1}
I20260806 08:58:00.115957 1885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63819ad3bc4acd9f939fcaaad62d66" candidate_uuid: "66adedc5560f4d43bec6a4a9c1323977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" is_pre_election: true
I20260806 08:58:00.116405 1885 raft_consensus.cc:2454] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66adedc5560f4d43bec6a4a9c1323977 in term 0.
I20260806 08:58:00.116750 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63819ad3bc4acd9f939fcaaad62d66" candidate_uuid: "66adedc5560f4d43bec6a4a9c1323977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963" is_pre_election: true
I20260806 08:58:00.117018 1707 raft_consensus.cc:2454] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66adedc5560f4d43bec6a4a9c1323977 in term 0.
I20260806 08:58:00.117096 1436 leader_election.cc:302] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:00.117758 1993 raft_consensus.cc:2781] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:58:00.117864 1993 raft_consensus.cc:482] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:58:00.117976 1993 raft_consensus.cc:3037] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.120255 1993 raft_consensus.cc:504] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.120954 1993 leader_election.cc:288] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [CANDIDATE]: Term 1 election: Requested vote from peers 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263), 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463)
I20260806 08:58:00.121472 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63819ad3bc4acd9f939fcaaad62d66" candidate_uuid: "66adedc5560f4d43bec6a4a9c1323977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963"
I20260806 08:58:00.121587 1885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db63819ad3bc4acd9f939fcaaad62d66" candidate_uuid: "66adedc5560f4d43bec6a4a9c1323977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9abe67e53e0b4eb7adb40bd8b700268a"
I20260806 08:58:00.121703 1707 raft_consensus.cc:3037] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.121826 1885 raft_consensus.cc:3037] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.124696 1885 raft_consensus.cc:2454] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66adedc5560f4d43bec6a4a9c1323977 in term 1.
I20260806 08:58:00.124749 1707 raft_consensus.cc:2454] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66adedc5560f4d43bec6a4a9c1323977 in term 1.
I20260806 08:58:00.125151 1436 leader_election.cc:302] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:00.125455 1993 raft_consensus.cc:2781] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:00.125962 1993 raft_consensus.cc:686] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 1 LEADER]: Becoming Leader. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Running, Role: LEADER
I20260806 08:58:00.126399 1993 consensus_queue.cc:237] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.129951 1314 catalog_manager.cc:5495] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1). New cstate: current_term: 1 leader_uuid: "66adedc5560f4d43bec6a4a9c1323977" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } health_report { overall_health: UNKNOWN } } }
Aug 06 08:58:00 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006677, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.62@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:58:00.236127 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.218229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41723 (local address 127.0.0.1:39184)
0806 08:58:00.219087 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.219108 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.219246 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.219618 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.219622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.220383 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:00.220560 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.220569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.221949 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.221953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.222866 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.222878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.223885 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.223890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.224030 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.227151 (+ 3121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:00.227170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:00.229474 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:00.233484 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:00.233498 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:00.233505 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:00.233948 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:00.234385 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:00.234389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:00.234391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:00.234542 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:00.235219 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:00.235226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:00.235404 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.235594 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.235843 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":606,"threads_started":1}
I20260806 08:58:00.236483 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.218229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.22.98.62:41723)
0806 08:58:00.218933 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.218940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.219206 (+ 266us) server_negotiation.cc:407] Connection header received
0806 08:58:00.219356 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.219361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.219445 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.219604 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:00.220721 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.221796 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.223012 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.223716 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.223852 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.229731 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:00.229739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:00.229744 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:00.229786 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:00.233320 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:00.234080 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:00.234086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:00.234090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:00.234263 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:00.234666 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:00.234672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:00.234675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:00.235071 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:00.235230 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.235710 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.236022 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":547,"thread_start_us":378,"threads_started":1}
I20260806 08:58:00.242064 1314 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:39184:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 08:58:00.243253 1314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:58:00.251113 1679 tablet_service.cc:1467] Processing CreateTablet for tablet f431a9a7afca46028798a4fef0cdf4fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dfc6e45f30ba4f71b32b92c56b57c725]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:58:00.251113 1494 tablet_service.cc:1467] Processing CreateTablet for tablet f431a9a7afca46028798a4fef0cdf4fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dfc6e45f30ba4f71b32b92c56b57c725]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:58:00.251798 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a9a7afca46028798a4fef0cdf4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.251812 1494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a9a7afca46028798a4fef0cdf4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.257776 1861 tablet_service.cc:1467] Processing CreateTablet for tablet f431a9a7afca46028798a4fef0cdf4fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dfc6e45f30ba4f71b32b92c56b57c725]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 08:58:00.258373 1861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a9a7afca46028798a4fef0cdf4fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.259187 1982 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: Bootstrap starting.
I20260806 08:58:00.259197 1981 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: Bootstrap starting.
I20260806 08:58:00.260955 1981 tablet_bootstrap.cc:654] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.260974 1982 tablet_bootstrap.cc:654] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.265798 1983 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: Bootstrap starting.
I20260806 08:58:00.267624 1983 tablet_bootstrap.cc:654] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.270049 1982 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: No bootstrap required, opened a new log
I20260806 08:58:00.270249 1982 ts_tablet_manager.cc:1397] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260806 08:58:00.270268 1981 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: No bootstrap required, opened a new log
I20260806 08:58:00.270431 1981 ts_tablet_manager.cc:1397] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260806 08:58:00.270980 1982 raft_consensus.cc:348] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.271189 1982 raft_consensus.cc:374] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.271266 1982 raft_consensus.cc:729] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.271188 1981 raft_consensus.cc:348] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.271430 1981 raft_consensus.cc:374] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.271503 1981 raft_consensus.cc:729] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679d49b23e374f5fb076cba3735e6963, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.271507 1982 consensus_queue.cc:260] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.271783 1981 consensus_queue.cc:260] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.272106 1982 ts_tablet_manager.cc:1428] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:58:00.272365 1981 ts_tablet_manager.cc:1428] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:58:00.272294 1989 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.272475 1993 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.272689 1989 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.272732 1993 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.274149 1983 tablet_bootstrap.cc:492] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: No bootstrap required, opened a new log
I20260806 08:58:00.274329 1983 ts_tablet_manager.cc:1397] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260806 08:58:00.275099 1983 raft_consensus.cc:348] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.275345 1983 raft_consensus.cc:374] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.275430 1983 raft_consensus.cc:729] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.275719 1983 consensus_queue.cc:260] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.276336 1983 ts_tablet_manager.cc:1428] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:58:00.276741 1995 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.277071 1995 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.503438 1995 raft_consensus.cc:482] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:58:00.503666 1995 raft_consensus.cc:504] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.504319 1995 leader_election.cc:288] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:00.504897 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a9a7afca46028798a4fef0cdf4fb" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977" is_pre_election: true
I20260806 08:58:00.504942 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a9a7afca46028798a4fef0cdf4fb" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963" is_pre_election: true
I20260806 08:58:00.505115 1524 raft_consensus.cc:2454] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:00.505196 1707 raft_consensus.cc:2454] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:00.505578 1793 leader_election.cc:302] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:00.505934 1995 raft_consensus.cc:2781] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:58:00.506034 1995 raft_consensus.cc:482] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:58:00.506116 1995 raft_consensus.cc:3037] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.508266 1995 raft_consensus.cc:504] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.508849 1995 leader_election.cc:288] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Requested vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:00.509325 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a9a7afca46028798a4fef0cdf4fb" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977"
I20260806 08:58:00.509358 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a9a7afca46028798a4fef0cdf4fb" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963"
I20260806 08:58:00.509519 1524 raft_consensus.cc:3037] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.509567 1707 raft_consensus.cc:3037] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:00.511584 1524 raft_consensus.cc:2454] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:00.511673 1707 raft_consensus.cc:2454] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:00.512089 1793 leader_election.cc:302] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:00.512368 1995 raft_consensus.cc:2781] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:00.512714 1995 raft_consensus.cc:686] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Becoming Leader. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Running, Role: LEADER
I20260806 08:58:00.512956 1995 consensus_queue.cc:237] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.514426 2040 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: TxnStatusTablet state changed. Reason: New leader 9abe67e53e0b4eb7adb40bd8b700268a. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.514711 2040 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:58:00.515316 2042 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 08:58:00.515568 2042 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 08:58:00.516024 1314 catalog_manager.cc:5495] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3). New cstate: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } health_report { overall_health: HEALTHY } } }
I20260806 08:58:00.552151 1993 consensus_queue.cc:1035] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:00.561245 2025 consensus_queue.cc:1035] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [LEADER]: Connected to new peer: Peer: permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:00.577629 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.569769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45463 (local address 127.0.0.1:34336)
0806 08:58:00.570002 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.570024 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.570188 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.570899 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.570905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.570919 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:58:00.571156 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.571167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.572851 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.572858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.574370 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.574386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.576171 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.576177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.576396 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.576461 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.576971 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.577005 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.577082 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.577245 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 08:58:00.577911 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.569292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.22.98.3:45463)
0806 08:58:00.569513 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.569521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.570401 (+ 880us) server_negotiation.cc:407] Connection header received
0806 08:58:00.570523 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.570529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.570623 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.570703 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:00.571340 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.572649 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.575214 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.575899 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.576034 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.576689 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.576832 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.576915 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.577579 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.577664 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 08:58:00.581588 1707 raft_consensus.cc:1261] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:00.581890 1524 raft_consensus.cc:1261] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:00.582192 2040 consensus_queue.cc:1035] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:00.582545 1995 consensus_queue.cc:1035] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:00.595388 1993 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: TxnStatusTablet state changed. Reason: New leader 9abe67e53e0b4eb7adb40bd8b700268a. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.595687 1993 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.597421 1989 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: TxnStatusTablet state changed. Reason: New leader 9abe67e53e0b4eb7adb40bd8b700268a. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.597747 1989 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.599822 2046 mvcc.cc:205] Tried to move back new op lower bound from 7315483363656720384 to 7315483363383267328. Current Snapshot: MvccSnapshot[applied={T|T < 7315483363656720384}]
I20260806 08:58:00.605616 2053 mvcc.cc:205] Tried to move back new op lower bound from 7315483363656720384 to 7315483363383267328. Current Snapshot: MvccSnapshot[applied={T|T < 7315483363656720384}]
I20260806 08:58:00.610473 1993 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.610744 1993 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 66adedc5560f4d43bec6a4a9c1323977: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.613536 1989 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.613909 1989 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 679d49b23e374f5fb076cba3735e6963: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 08:58:00.627207 1995 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.627656 1995 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:58:00.630331 1995 tablet_replica.cc:439] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } } }
I20260806 08:58:00.630637 1995 tablet_replica.cc:442] T f431a9a7afca46028798a4fef0cdf4fb P 9abe67e53e0b4eb7adb40bd8b700268a: This TxnStatusTablet replica's current role is: LEADER
I20260806 08:58:00.644656 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.623284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.22.98.1:43263)
0806 08:58:00.639106 (+ 15822us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.639111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.639129 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:58:00.639225 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.639230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.639310 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.639477 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:00.640113 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.640835 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.642195 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.642615 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.642728 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.643291 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.644001 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.644153 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.644261 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.644353 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15638,"thread_start_us":15531,"threads_started":1}
I20260806 08:58:00.661091 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.654963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:36467 (local address 127.22.98.3:45463)
0806 08:58:00.655188 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.655195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.656177 (+ 982us) server_negotiation.cc:407] Connection header received
0806 08:58:00.656398 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.656403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.656476 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.656552 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:00.657003 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.657821 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.659184 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.659721 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.659843 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.660258 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.660395 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.660479 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.660648 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.660746 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 08:58:00.661144 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.654681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45463 (local address 127.22.98.1:36467)
0806 08:58:00.656067 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.656085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.656255 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.656583 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.656588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.656600 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:58:00.656855 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.656865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.657963 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.657968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.659038 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.659052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.659878 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.659884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.660021 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.660082 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.660492 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.660524 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.660606 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.660772 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"thread_start_us":1056,"threads_started":1}
I20260806 08:58:00.702809 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.695390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43263 (local address 127.22.98.3:34211)
0806 08:58:00.696301 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:00.696316 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:00.696460 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:00.696795 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:00.696799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:00.696811 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:58:00.697154 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.697163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.698479 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.698486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.700381 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:00.700400 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.701291 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.701299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:00.701481 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.701569 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.702109 (+ 540us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.702122 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 08:58:00.702204 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 08:58:00.702368 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":591,"threads_started":1}
I20260806 08:58:00.703258 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.695789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:34211 (local address 127.22.98.1:43263)
0806 08:58:00.695948 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.695954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.696400 (+ 446us) server_negotiation.cc:407] Connection header received
0806 08:58:00.696595 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.696600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.696670 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.696777 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:00.697312 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.698324 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.700555 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.701079 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.701203 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.701811 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:00.701959 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:00.702049 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.703005 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.703075 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 08:58:00.741765 1315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:00.742050 1315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:00.745790 1315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Sending DeleteTablet for 3 replicas of tablet db63819ad3bc4acd9f939fcaaad62d66
I20260806 08:58:00.746853 1494 tablet_service.cc:1514] Processing DeleteTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:00 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57958
I20260806 08:58:00.746956 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:00 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55760
I20260806 08:58:00.747875 1861 tablet_service.cc:1514] Processing DeleteTablet for tablet db63819ad3bc4acd9f939fcaaad62d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:00 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34308
I20260806 08:58:00.757022 2075 tablet_replica.cc:330] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: stopping tablet replica
I20260806 08:58:00.757422 2075 raft_consensus.cc:2229] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:58:00.758129 2075 raft_consensus.cc:2258] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:00.759366 2075 ts_tablet_manager.cc:1905] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:00.766990 2075 ts_tablet_manager.cc:1918] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:00.767230 2075 log.cc:1196] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/db63819ad3bc4acd9f939fcaaad62d66
I20260806 08:58:00.767833 2075 ts_tablet_manager.cc:1939] T db63819ad3bc4acd9f939fcaaad62d66 P 66adedc5560f4d43bec6a4a9c1323977: Deleting consensus metadata
I20260806 08:58:00.768699 2078 tablet_replica.cc:330] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: stopping tablet replica
I20260806 08:58:00.768932 2078 raft_consensus.cc:2229] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:00.769147 1293 catalog_manager.cc:4841] TS 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263): tablet db63819ad3bc4acd9f939fcaaad62d66 (table test-table [id=de7f94535a504b6f85af8dc6621feefb]) successfully deleted
I20260806 08:58:00.772640 2078 raft_consensus.cc:2258] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:00.774641 2078 ts_tablet_manager.cc:1905] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:00.775290 2077 tablet_replica.cc:330] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: stopping tablet replica
I20260806 08:58:00.775517 2077 raft_consensus.cc:2229] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:00.775897 2077 raft_consensus.cc:2258] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:00.777091 2077 ts_tablet_manager.cc:1905] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:00.784345 2078 ts_tablet_manager.cc:1918] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:00.784566 2078 log.cc:1196] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/db63819ad3bc4acd9f939fcaaad62d66
I20260806 08:58:00.785144 2078 ts_tablet_manager.cc:1939] T db63819ad3bc4acd9f939fcaaad62d66 P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting consensus metadata
I20260806 08:58:00.786564 1295 catalog_manager.cc:4841] TS 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463): tablet db63819ad3bc4acd9f939fcaaad62d66 (table test-table [id=de7f94535a504b6f85af8dc6621feefb]) successfully deleted
I20260806 08:58:00.789752 2077 ts_tablet_manager.cc:1918] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:00.789935 2077 log.cc:1196] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/db63819ad3bc4acd9f939fcaaad62d66
I20260806 08:58:00.792146 2077 ts_tablet_manager.cc:1939] T db63819ad3bc4acd9f939fcaaad62d66 P 679d49b23e374f5fb076cba3735e6963: Deleting consensus metadata
I20260806 08:58:00.793442 1297 catalog_manager.cc:4841] TS 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263): tablet db63819ad3bc4acd9f939fcaaad62d66 (table test-table [id=de7f94535a504b6f85af8dc6621feefb]) successfully deleted
Aug 06 08:58:00 dist-test-slave-jpj1 krb5kdc[1259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 08:58:00.851332 22920 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.102s user 0.003s sys 0.021s
Aug 06 08:58:00 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:00 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 08:58:00.885030 2087 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:00.855063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.22.98.62:41723)
0806 08:58:00.870826 (+ 15763us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.870832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.870854 (+ 22us) server_negotiation.cc:407] Connection header received
0806 08:58:00.870920 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.870926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.871007 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.871128 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:00.873200 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.874405 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.876286 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.876950 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.877081 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.884728 (+ 7647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38754: BlockingRecv error: recv got EOF from 127.0.0.1:38754 (error 108)
Metrics: {"server-negotiator.queue_time_us":15616,"thread_start_us":15470,"threads_started":1}
Aug 06 08:58:00 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM
I20260806 08:58:00.906497 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:00.888270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.22.98.62:41723)
0806 08:58:00.888441 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:00.888447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:00.888743 (+ 296us) server_negotiation.cc:407] Connection header received
0806 08:58:00.889031 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:00.889037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:00.889121 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:00.889227 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:00.890883 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.892046 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.893930 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:00.894570 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:00.894707 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:00.899792 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:00.899800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:00.899806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:00.899853 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:00.903439 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:00.904314 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:00.904320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:00.904324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:00.904486 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:00.904921 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:00.904926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:00.904928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:00.905325 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:00.905601 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:00.905895 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 08:58:00.906146 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 08:58:00.911351 1315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:58:00.912137 1315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:58:00.920223 1494 tablet_service.cc:1467] Processing CreateTablet for tablet 44e678b1141649a4ab5745115c5010fa (DEFAULT_TABLE table=test-table [id=e8d57d08f83f4036869235f544c57be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:00.920742 1494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e678b1141649a4ab5745115c5010fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.921202 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 44e678b1141649a4ab5745115c5010fa (DEFAULT_TABLE table=test-table [id=e8d57d08f83f4036869235f544c57be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:00.921634 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e678b1141649a4ab5745115c5010fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.921519 1861 tablet_service.cc:1467] Processing CreateTablet for tablet 44e678b1141649a4ab5745115c5010fa (DEFAULT_TABLE table=test-table [id=e8d57d08f83f4036869235f544c57be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:00.922013 1861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e678b1141649a4ab5745115c5010fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:00.932587 2094 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Bootstrap starting.
I20260806 08:58:00.933475 2095 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Bootstrap starting.
I20260806 08:58:00.934037 2093 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Bootstrap starting.
I20260806 08:58:00.936328 2093 tablet_bootstrap.cc:654] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.936699 2094 tablet_bootstrap.cc:654] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.938552 2095 tablet_bootstrap.cc:654] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:00.948199 2094 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: No bootstrap required, opened a new log
I20260806 08:58:00.948452 2094 ts_tablet_manager.cc:1397] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260806 08:58:00.949010 2095 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: No bootstrap required, opened a new log
I20260806 08:58:00.949280 2095 ts_tablet_manager.cc:1397] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260806 08:58:00.949452 2094 raft_consensus.cc:348] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.949836 2094 raft_consensus.cc:374] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.949997 2094 raft_consensus.cc:729] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679d49b23e374f5fb076cba3735e6963, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.950277 2095 raft_consensus.cc:348] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.950352 2094 consensus_queue.cc:260] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.951591 2095 raft_consensus.cc:374] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.951769 2095 raft_consensus.cc:729] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.952680 2093 tablet_bootstrap.cc:492] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: No bootstrap required, opened a new log
I20260806 08:58:00.952919 2093 ts_tablet_manager.cc:1397] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260806 08:58:00.953930 2093 raft_consensus.cc:348] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.954303 2093 raft_consensus.cc:374] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:00.954474 2093 raft_consensus.cc:729] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Initialized, Role: FOLLOWER
I20260806 08:58:00.954923 2094 ts_tablet_manager.cc:1428] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 08:58:00.954839 2093 consensus_queue.cc:260] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.955484 2093 ts_tablet_manager.cc:1428] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:58:00.956015 2095 consensus_queue.cc:260] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:00.957762 2095 ts_tablet_manager.cc:1428] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 08:58:01.238119 2059 raft_consensus.cc:482] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:58:01.238373 2059 raft_consensus.cc:504] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.239142 2059 leader_election.cc:288] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:01.239866 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e678b1141649a4ab5745115c5010fa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963" is_pre_election: true
I20260806 08:58:01.239859 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e678b1141649a4ab5745115c5010fa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977" is_pre_election: true
I20260806 08:58:01.240150 1524 raft_consensus.cc:2454] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:01.240172 1707 raft_consensus.cc:2454] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:01.240676 1793 leader_election.cc:302] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:01.240974 2059 raft_consensus.cc:2781] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:58:01.241106 2059 raft_consensus.cc:482] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:58:01.241205 2059 raft_consensus.cc:3037] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.243335 2059 raft_consensus.cc:504] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.243964 2059 leader_election.cc:288] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Requested vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:01.244524 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e678b1141649a4ab5745115c5010fa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977"
I20260806 08:58:01.244566 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e678b1141649a4ab5745115c5010fa" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963"
I20260806 08:58:01.244769 1524 raft_consensus.cc:3037] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.244817 1707 raft_consensus.cc:3037] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.247872 1707 raft_consensus.cc:2454] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:01.247891 1524 raft_consensus.cc:2454] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:01.248402 1793 leader_election.cc:302] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:01.248723 2059 raft_consensus.cc:2781] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:01.249090 2059 raft_consensus.cc:686] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Becoming Leader. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Running, Role: LEADER
I20260806 08:58:01.249369 2059 consensus_queue.cc:237] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.252470 1315 catalog_manager.cc:5495] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3). New cstate: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } health_report { overall_health: HEALTHY } } }
I20260806 08:58:01.299935 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:01.295046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.22.98.3:45463)
0806 08:58:01.295503 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:01.295508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:01.295522 (+ 14us) server_negotiation.cc:407] Connection header received
0806 08:58:01.295616 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:01.295620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:01.295685 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:01.295772 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:01.296169 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.296868 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.298187 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.298605 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.298719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:01.299130 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:01.299237 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:01.299312 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:01.299561 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 08:58:01.299644 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260806 08:58:01.302866 1524 raft_consensus.cc:1261] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:01.302877 1707 raft_consensus.cc:1261] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:01.303563 2100 consensus_queue.cc:1035] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:01.303903 2059 consensus_queue.cc:1035] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:01.319090 1314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:01.319370 1314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:01.324528 1314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Sending DeleteTablet for 3 replicas of tablet 44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.325493 1861 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e678b1141649a4ab5745115c5010fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34308
I20260806 08:58:01.325660 1494 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e678b1141649a4ab5745115c5010fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57958
I20260806 08:58:01.325773 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e678b1141649a4ab5745115c5010fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34308
I20260806 08:58:01.326002 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e678b1141649a4ab5745115c5010fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55760
I20260806 08:58:01.326002 1493 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e678b1141649a4ab5745115c5010fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57958
I20260806 08:58:01.326392 2119 tablet_replica.cc:330] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: stopping tablet replica
I20260806 08:58:01.326613 2119 raft_consensus.cc:2229] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:01.326639 2120 tablet_replica.cc:330] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: stopping tablet replica
I20260806 08:58:01.326845 2120 raft_consensus.cc:2229] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:01.326920 2119 raft_consensus.cc:2258] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.327162 2120 raft_consensus.cc:2258] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.328540 2120 ts_tablet_manager.cc:1905] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.328526 2119 ts_tablet_manager.cc:1905] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.328521 1313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:58:01.329169 1313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:58:01.335150 2120 ts_tablet_manager.cc:1918] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:01.335362 2120 log.cc:1196] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.335629 2119 ts_tablet_manager.cc:1918] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:01.335815 2119 log.cc:1196] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.335930 2120 ts_tablet_manager.cc:1939] T 44e678b1141649a4ab5745115c5010fa P 679d49b23e374f5fb076cba3735e6963: Deleting consensus metadata
I20260806 08:58:01.336303 2119 ts_tablet_manager.cc:1939] T 44e678b1141649a4ab5745115c5010fa P 66adedc5560f4d43bec6a4a9c1323977: Deleting consensus metadata
I20260806 08:58:01.336305 2118 tablet_replica.cc:330] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: stopping tablet replica
I20260806 08:58:01.336508 2118 raft_consensus.cc:2229] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:58:01.337045 2118 raft_consensus.cc:2258] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.337064 1297 catalog_manager.cc:4841] TS 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263): tablet 44e678b1141649a4ab5745115c5010fa (table test-table [id=e8d57d08f83f4036869235f544c57be5]) successfully deleted
I20260806 08:58:01.337769 1679 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (DEFAULT_TABLE table=test-table [id=5e373bd5cbf3417189ed627a30d481c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:01.337785 1861 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (DEFAULT_TABLE table=test-table [id=5e373bd5cbf3417189ed627a30d481c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:01.338191 1861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:01.338263 1679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:01.340466 2118 ts_tablet_manager.cc:1905] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.340664 1293 catalog_manager.cc:4841] TS 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263): tablet 44e678b1141649a4ab5745115c5010fa (table test-table [id=e8d57d08f83f4036869235f544c57be5]) successfully deleted
W20260806 08:58:01.341171 1293 catalog_manager.cc:4808] TS 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263): delete failed for tablet 44e678b1141649a4ab5745115c5010fa because the tablet was not found. No further retry: Not found: Tablet not found: 44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.342008 1494 tablet_service.cc:1467] Processing CreateTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (DEFAULT_TABLE table=test-table [id=5e373bd5cbf3417189ed627a30d481c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:01.342499 1494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:01.344890 2094 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Bootstrap starting.
I20260806 08:58:01.344935 2095 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Bootstrap starting.
I20260806 08:58:01.346326 2094 tablet_bootstrap.cc:654] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:01.346705 2095 tablet_bootstrap.cc:654] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:01.348413 2118 ts_tablet_manager.cc:1918] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:01.348619 2118 log.cc:1196] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.349505 2094 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: No bootstrap required, opened a new log
I20260806 08:58:01.349766 2094 ts_tablet_manager.cc:1397] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 08:58:01.349946 2118 ts_tablet_manager.cc:1939] T 44e678b1141649a4ab5745115c5010fa P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting consensus metadata
I20260806 08:58:01.350553 2094 raft_consensus.cc:348] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.350878 2094 raft_consensus.cc:374] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:01.351011 2094 raft_consensus.cc:729] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679d49b23e374f5fb076cba3735e6963, State: Initialized, Role: FOLLOWER
I20260806 08:58:01.351338 2094 consensus_queue.cc:260] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.351745 1295 catalog_manager.cc:4841] TS 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463): tablet 44e678b1141649a4ab5745115c5010fa (table test-table [id=e8d57d08f83f4036869235f544c57be5]) successfully deleted
I20260806 08:58:01.352053 2094 ts_tablet_manager.cc:1428] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260806 08:58:01.352335 1295 catalog_manager.cc:4808] TS 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463): delete failed for tablet 44e678b1141649a4ab5745115c5010fa because the tablet was not found. No further retry: Not found: Tablet not found: 44e678b1141649a4ab5745115c5010fa
I20260806 08:58:01.353408 2093 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Bootstrap starting.
I20260806 08:58:01.354578 2095 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: No bootstrap required, opened a new log
I20260806 08:58:01.354735 2095 ts_tablet_manager.cc:1397] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260806 08:58:01.355077 2093 tablet_bootstrap.cc:654] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:01.355520 2095 raft_consensus.cc:348] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.355818 2095 raft_consensus.cc:374] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:01.355895 2095 raft_consensus.cc:729] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Initialized, Role: FOLLOWER
I20260806 08:58:01.356184 2095 consensus_queue.cc:260] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.356842 2095 ts_tablet_manager.cc:1428] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 08:58:01.357702 2093 tablet_bootstrap.cc:492] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: No bootstrap required, opened a new log
I20260806 08:58:01.357868 2093 ts_tablet_manager.cc:1397] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 08:58:01.358515 2093 raft_consensus.cc:348] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.358700 2093 raft_consensus.cc:374] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:01.358788 2093 raft_consensus.cc:729] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66adedc5560f4d43bec6a4a9c1323977, State: Initialized, Role: FOLLOWER
I20260806 08:58:01.359007 2093 consensus_queue.cc:260] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.359618 2093 ts_tablet_manager.cc:1428] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 08:58:01.542327 2100 raft_consensus.cc:482] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:58:01.542572 2100 raft_consensus.cc:504] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.543236 2100 leader_election.cc:288] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:01.543998 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdc7c1f90534d9ebd5441eefa98ddbe" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963" is_pre_election: true
I20260806 08:58:01.544021 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdc7c1f90534d9ebd5441eefa98ddbe" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977" is_pre_election: true
I20260806 08:58:01.544272 1524 raft_consensus.cc:2454] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:01.544314 1707 raft_consensus.cc:2454] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 0.
I20260806 08:58:01.544759 1799 leader_election.cc:302] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 66adedc5560f4d43bec6a4a9c1323977, 679d49b23e374f5fb076cba3735e6963, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:01.545104 2100 raft_consensus.cc:2781] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:58:01.545199 2100 raft_consensus.cc:482] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:58:01.545305 2100 raft_consensus.cc:3037] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.547573 2100 raft_consensus.cc:504] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.548157 2100 leader_election.cc:288] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Requested vote from peers 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263), 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263)
I20260806 08:58:01.548661 1707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdc7c1f90534d9ebd5441eefa98ddbe" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679d49b23e374f5fb076cba3735e6963"
I20260806 08:58:01.548872 1707 raft_consensus.cc:3037] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.551031 1707 raft_consensus.cc:2454] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:01.551470 1799 leader_election.cc:302] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 679d49b23e374f5fb076cba3735e6963, 9abe67e53e0b4eb7adb40bd8b700268a; no voters:
I20260806 08:58:01.551441 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fdc7c1f90534d9ebd5441eefa98ddbe" candidate_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66adedc5560f4d43bec6a4a9c1323977"
I20260806 08:58:01.551672 1524 raft_consensus.cc:3037] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:01.551774 2100 raft_consensus.cc:2781] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:01.551946 2100 raft_consensus.cc:686] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Becoming Leader. State: Replica: 9abe67e53e0b4eb7adb40bd8b700268a, State: Running, Role: LEADER
I20260806 08:58:01.552211 2100 consensus_queue.cc:237] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } }
I20260806 08:58:01.554339 1524 raft_consensus.cc:2454] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abe67e53e0b4eb7adb40bd8b700268a in term 1.
I20260806 08:58:01.555070 1314 catalog_manager.cc:5495] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3). New cstate: current_term: 1 leader_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9abe67e53e0b4eb7adb40bd8b700268a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45463 } health_report { overall_health: HEALTHY } } }
I20260806 08:58:01.622133 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:01.617177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:41163 (local address 127.22.98.3:45463)
0806 08:58:01.617329 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:01.617334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:01.617767 (+ 433us) server_negotiation.cc:407] Connection header received
0806 08:58:01.617930 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:01.617933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:01.617998 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:01.618071 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:01.618513 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.619290 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.620620 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.621010 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.621101 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:01.621497 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:01.621648 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:01.621729 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:01.621823 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 08:58:01.621886 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":44}
I20260806 08:58:01.622215 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:01.617110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45463 (local address 127.22.98.3:41163)
0806 08:58:01.617682 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:01.617698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:01.617824 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:01.618096 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:01.618099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:01.618108 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 08:58:01.618367 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:01.618375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.619415 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.619419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:01.620501 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:01.620515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.621164 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.621169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:01.621293 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:01.621330 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 08:58:01.621730 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 08:58:01.621737 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 08:58:01.621798 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 08:58:01.621893 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":14,"thread_start_us":273,"threads_started":1}
I20260806 08:58:01.630069 1524 raft_consensus.cc:1261] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:01.630069 1707 raft_consensus.cc:1261] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Refusing update from remote peer 9abe67e53e0b4eb7adb40bd8b700268a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:01.630628 2100 consensus_queue.cc:1035] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "679d49b23e374f5fb076cba3735e6963" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:01.630893 2059 consensus_queue.cc:1035] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66adedc5560f4d43bec6a4a9c1323977" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:01.635792 2106 mvcc.cc:205] Tried to move back new op lower bound from 7315483367952392192 to 7315483367640702976. Current Snapshot: MvccSnapshot[applied={T|T < 7315483367952392192}]
I20260806 08:58:01.681974 1313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:01.682199 1313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:01.685393 1313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a208de19fb1841b9b0faedae07933b73: Sending DeleteTablet for 3 replicas of tablet 1fdc7c1f90534d9ebd5441eefa98ddbe
I20260806 08:58:01.686348 1494 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57958
I20260806 08:58:01.686405 1679 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55760
I20260806 08:58:01.686720 2120 tablet_replica.cc:330] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: stopping tablet replica
I20260806 08:58:01.686758 2119 tablet_replica.cc:330] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: stopping tablet replica
I20260806 08:58:01.686870 1862 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fdc7c1f90534d9ebd5441eefa98ddbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:01 UTC) from {username='oryx', principal='oryx/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34308
I20260806 08:58:01.686945 2120 raft_consensus.cc:2229] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:01.686968 2119 raft_consensus.cc:2229] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:01.687145 2120 raft_consensus.cc:2258] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.687155 2119 raft_consensus.cc:2258] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.687325 2118 tablet_replica.cc:330] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: stopping tablet replica
I20260806 08:58:01.687521 2118 raft_consensus.cc:2229] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:58:01.687780 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 1413
I20260806 08:58:01.687893 2118 raft_consensus.cc:2258] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:01.688675 2119 ts_tablet_manager.cc:1905] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.689105 2120 ts_tablet_manager.cc:1905] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.695964 2119 ts_tablet_manager.cc:1918] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:01.696198 2119 log.cc:1196] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/1fdc7c1f90534d9ebd5441eefa98ddbe
I20260806 08:58:01.696661 2119 ts_tablet_manager.cc:1939] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 66adedc5560f4d43bec6a4a9c1323977: Deleting consensus metadata
I20260806 08:58:01.698009 1293 catalog_manager.cc:4841] TS 66adedc5560f4d43bec6a4a9c1323977 (127.22.98.1:43263): tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (table test-table [id=5e373bd5cbf3417189ed627a30d481c2]) successfully deleted
I20260806 08:58:01.699672 2118 ts_tablet_manager.cc:1905] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:01.699977 2120 ts_tablet_manager.cc:1918] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:01.700139 2120 log.cc:1196] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/1fdc7c1f90534d9ebd5441eefa98ddbe
I20260806 08:58:01.700510 2120 ts_tablet_manager.cc:1939] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 679d49b23e374f5fb076cba3735e6963: Deleting consensus metadata
I20260806 08:58:01.701526 1297 catalog_manager.cc:4841] TS 679d49b23e374f5fb076cba3735e6963 (127.22.98.2:36263): tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (table test-table [id=5e373bd5cbf3417189ed627a30d481c2]) successfully deleted
I20260806 08:58:01.705724 2118 ts_tablet_manager.cc:1918] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 08:58:01.705880 2118 log.cc:1196] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/1fdc7c1f90534d9ebd5441eefa98ddbe
I20260806 08:58:01.706234 2118 ts_tablet_manager.cc:1939] T 1fdc7c1f90534d9ebd5441eefa98ddbe P 9abe67e53e0b4eb7adb40bd8b700268a: Deleting consensus metadata
I20260806 08:58:01.707218 1295 catalog_manager.cc:4841] TS 9abe67e53e0b4eb7adb40bd8b700268a (127.22.98.3:45463): tablet 1fdc7c1f90534d9ebd5441eefa98ddbe (table test-table [id=5e373bd5cbf3417189ed627a30d481c2]) successfully deleted
Aug 06 08:58:01 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.1@KRBTEST.COM
I20260806 08:58:01.714488 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:01.689042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.22.98.1:43263)
0806 08:58:01.698944 (+ 9902us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:01.698948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:01.698966 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:58:01.699015 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:01.699019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:01.699082 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:01.699168 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:01.701107 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.702187 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.703462 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.704205 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.704338 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:01.708605 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:01.708627 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:01.708651 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:01.708705 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:01.711908 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:01.712588 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:01.712597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:01.712599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:01.712740 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:01.713070 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:01.713073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:01.713075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:01.713452 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:01.713692 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:01.713955 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 08:58:01.714129 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9788,"thread_start_us":9656,"threads_started":1}
I20260806 08:58:01.715143 1585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:01.849877 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 1413
I20260806 08:58:01.879891 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 1596
Aug 06 08:58:01 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.2@KRBTEST.COM
I20260806 08:58:01.895635 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:01.880660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.22.98.2:36263)
0806 08:58:01.881133 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:01.881138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:01.881156 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:58:01.881244 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:01.881248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:01.881311 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:01.881400 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:01.882582 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.883321 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.884940 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:01.885423 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:01.885528 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:01.890015 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:01.890031 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:01.890050 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:01.890092 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:01.893000 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:01.893643 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:01.893650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:01.893652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:01.893802 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:01.894126 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:01.894129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:01.894131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:01.894498 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:01.894714 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:01.895088 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 08:58:01.895256 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":257,"threads_started":1}
I20260806 08:58:01.896371 1758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:01.992825 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 1596
I20260806 08:58:02.018460 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 1772
Aug 06 08:58:02 dist-test-slave-jpj1 krb5kdc[1259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.98.3@KRBTEST.COM
I20260806 08:58:02.034914 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:02.019433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.22.98.3:45463)
0806 08:58:02.019707 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:02.019714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:02.019742 (+ 28us) server_negotiation.cc:407] Connection header received
0806 08:58:02.019813 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:02.019817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:02.019882 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:02.019967 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:02.021506 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:02.022303 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:02.023610 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:02.024108 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:02.024243 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:02.029177 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:02.029197 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:02.029218 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:02.029270 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:02.032289 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:02.033007 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:02.033017 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:02.033019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:02.033194 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:02.033517 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:02.033519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:02.033522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:02.033899 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:02.034154 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:02.034464 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 08:58:02.034651 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 08:58:02.035746 1943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:02.184387 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 1772
I20260806 08:58:02.216948 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 1275
I20260806 08:58:02.218212 1371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:02.336005 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 1275
2026-08-06T08:58:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6471 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:58:02 dist-test-slave-jpj1 krb5kdc[2154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:58:02 dist-test-slave-jpj1 krb5kdc[2154](info): set up 2 sockets
Aug 06 08:58:02 dist-test-slave-jpj1 krb5kdc[2154](info): commencing operation
krb5kdc: starting...
W20260806 08:58:03.541908 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:58:03 dist-test-slave-jpj1 krb5kdc[2154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:58:03.813452 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:45123
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:45123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:04.159420 2171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:04.159751 2171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:04.159852 2171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:04.167059 2171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:04.167184 2171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:04.167232 2171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:04.167294 2171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:04.167342 2171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:04.177393 2171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:45123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:04.178692 2171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:04.180122 2171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:04.192217 2177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:04.192256 2178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:04.192281 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:04.193019 2171 server_base.cc:1034] running on GCE node
I20260806 08:58:04.193869 2171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:04.195421 2171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:04.196707 2171 hybrid_clock.cc:648] HybridClock initialized: now 1786006684196643 us; error 53 us; skew 500 ppm
I20260806 08:58:04.197090 2171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T08:58:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1923 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:58:04 dist-test-slave-jpj1 krb5kdc[2188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:58:04 dist-test-slave-jpj1 krb5kdc[2188](info): set up 2 sockets
Aug 06 08:58:04 dist-test-slave-jpj1 krb5kdc[2188](info): commencing operation
krb5kdc: starting...
W20260806 08:58:06.078735 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.767s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:58:06 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:58:06.508399 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35903
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:07.163599 2207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:07.164269 2207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:07.164489 2207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:07.178597 2207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:07.178815 2207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:07.179000 2207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:07.179141 2207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:07.179265 2207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:07.198983 2207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:07.201678 2207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:07.204363 2207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:07.228036 2215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:07.227995 2216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:07.227969 2218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:07.231665 2207 server_base.cc:1034] running on GCE node
I20260806 08:58:07.233376 2207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:07.236243 2207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:07.237964 2207 hybrid_clock.cc:648] HybridClock initialized: now 1786006687237804 us; error 100 us; skew 500 ppm
I20260806 08:58:07.238801 2207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:07 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006687, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:07.248803 2207 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:58:07.252347 2207 webserver.cc:460] Webserver started at http://127.22.98.62:35875/ using document root <none> and password file <none>
I20260806 08:58:07.254016 2207 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:07.254297 2207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:07.254936 2207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:07.261899 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "a634f95aa59d4232a5867f68c0b69077"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "cb53356946e6d50a40808cc2b5c9ac91"
server_key_iv: "247e5a7ff74c6354da2fd0bd65f9eb1f"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.263651 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "a634f95aa59d4232a5867f68c0b69077"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "cb53356946e6d50a40808cc2b5c9ac91"
server_key_iv: "247e5a7ff74c6354da2fd0bd65f9eb1f"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.275606 2207 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.004s
I20260806 08:58:07.282814 2227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:07.285624 2207 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260806 08:58:07.285928 2207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "a634f95aa59d4232a5867f68c0b69077"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "cb53356946e6d50a40808cc2b5c9ac91"
server_key_iv: "247e5a7ff74c6354da2fd0bd65f9eb1f"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.286330 2207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:07.331740 2207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:07.336764 2207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:07.337061 2207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:07.356858 2207 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:35903
I20260806 08:58:07.356956 2289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:35903 every 8 connection(s)
I20260806 08:58:07.358315 2207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:58:07.359680 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2207
I20260806 08:58:07.359954 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:58:07.360785 22920 external_mini_cluster.cc:1428] Setting key e1791f436cccff206aaaa6e89fe386bb
I20260806 08:58:07.363037 2290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:07.377209 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Bootstrap starting.
Aug 06 08:58:07 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:07.381484 2290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:07.383014 2290 log.cc:824] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:07.386750 2290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: No bootstrap required, opened a new log
I20260806 08:58:07.387915 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:07.363160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.22.98.62:35903)
0806 08:58:07.364125 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:07.364142 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:07.364205 (+ 63us) server_negotiation.cc:407] Connection header received
0806 08:58:07.366468 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:07.366501 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:07.367083 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:07.367598 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:07.369406 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:07.371591 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:07.373047 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:07.374108 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:07.374328 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:07.378823 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:07.378839 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:07.378862 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:07.378919 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:07.383075 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:07.383915 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:07.383922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:07.383930 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:07.384106 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:07.384591 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:07.384597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:07.384600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:07.385356 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:07.385712 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:07.386020 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 08:58:07.386495 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":425,"threads_started":1}
I20260806 08:58:07.395583 2290 raft_consensus.cc:348] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:07.395957 2290 raft_consensus.cc:374] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:07.396034 2290 raft_consensus.cc:729] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a634f95aa59d4232a5867f68c0b69077, State: Initialized, Role: FOLLOWER
I20260806 08:58:07.396595 2290 consensus_queue.cc:260] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:07.396888 2290 raft_consensus.cc:388] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:58:07.397040 2290 raft_consensus.cc:482] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:58:07.397225 2290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:07.399487 2290 raft_consensus.cc:504] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:07.400132 2290 leader_election.cc:302] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a634f95aa59d4232a5867f68c0b69077; no voters:
I20260806 08:58:07.400811 2290 leader_election.cc:288] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:58:07.400964 2296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:07.401515 2296 raft_consensus.cc:686] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 LEADER]: Becoming Leader. State: Replica: a634f95aa59d4232a5867f68c0b69077, State: Running, Role: LEADER
I20260806 08:58:07.402045 2296 consensus_queue.cc:237] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:07.405076 2298 sys_catalog.cc:455] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a634f95aa59d4232a5867f68c0b69077. Latest consensus state: current_term: 1 leader_uuid: "a634f95aa59d4232a5867f68c0b69077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } } }
I20260806 08:58:07.405567 2290 sys_catalog.cc:564] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:58:07.405846 2298 sys_catalog.cc:458] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:07.406911 2296 sys_catalog.cc:455] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a634f95aa59d4232a5867f68c0b69077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } } }
I20260806 08:58:07.407125 2296 sys_catalog.cc:458] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:07.409049 2300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:58:07.415745 2300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:58:07.426600 2300 catalog_manager.cc:1347] Generated new cluster ID: b656ba3fd44449449fadd59923318857
I20260806 08:58:07.426880 2300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:58:07.455140 2300 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:58:07.457351 2300 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:58:07.466710 2300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Generated new TSK 0
I20260806 08:58:07.467970 2300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:58:07.536131 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:07.904817 2330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:07.905162 2330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:07.905318 2330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:07.914918 2330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:07.915045 2330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:07.915210 2330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:58:07.929332 2330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:07.931476 2330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:07.933228 2330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:07.950505 2340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:07.950809 2342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:07.950518 2339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:07.951604 2330 server_base.cc:1034] running on GCE node
I20260806 08:58:07.952430 2330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:07.953809 2330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:07.955029 2330 hybrid_clock.cc:648] HybridClock initialized: now 1786006687954970 us; error 57 us; skew 500 ppm
I20260806 08:58:07.955425 2330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:07 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006687, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:07.958714 2330 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:58:07.960371 2330 webserver.cc:460] Webserver started at http://127.22.98.1:34405/ using document root <none> and password file <none>
I20260806 08:58:07.961122 2330 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:07.961241 2330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:07.961537 2330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:07.964708 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "657401d72d994c7d8965989031fd5cfa"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "159761b2024ebdafcac9c8a7d70d91ff"
server_key_iv: "29c6d3fbbc25369dca9f8e62abbb1df4"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.965553 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "657401d72d994c7d8965989031fd5cfa"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "159761b2024ebdafcac9c8a7d70d91ff"
server_key_iv: "29c6d3fbbc25369dca9f8e62abbb1df4"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.971166 2330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:58:07.975055 2353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:07.976507 2330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:58:07.976671 2330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "657401d72d994c7d8965989031fd5cfa"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "159761b2024ebdafcac9c8a7d70d91ff"
server_key_iv: "29c6d3fbbc25369dca9f8e62abbb1df4"
server_key_version: "encryptionkey@0"
I20260806 08:58:07.976881 2330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:08.007795 2330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:08.012470 2330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:08.012763 2330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:08.013844 2330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:08.015503 2330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:08.015621 2330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 08:58:08.015762 2330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:08.015820 2330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:08.052024 2330 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:39667
I20260806 08:58:08.052131 2505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:39667 every 8 connection(s)
I20260806 08:58:08.053435 2330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:58:08.055717 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2330
I20260806 08:58:08.055976 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:58:08.056855 22920 external_mini_cluster.cc:1428] Setting key 3fbd4b9828649785e0e3e28dfd27bbd5
Aug 06 08:58:08 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006687, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:08.076053 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:08.056628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:48729 (local address 127.22.98.62:35903)
0806 08:58:08.057211 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:08.057218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:08.058277 (+ 1059us) server_negotiation.cc:407] Connection header received
0806 08:58:08.060276 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:08.060282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:08.060368 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:08.060464 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:08.062097 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.062851 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.064646 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.065045 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.065165 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:08.069171 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:08.069179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:08.069184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:08.069235 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:08.072082 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:08.073136 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:08.073141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:08.073143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:08.073407 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:08.073919 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:08.073922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:08.073924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:08.074213 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:08.074347 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:08.075447 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0806 08:58:08.075689 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":315,"threads_started":1}
I20260806 08:58:08.077521 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:08.057254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:48729)
0806 08:58:08.058110 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:08.058163 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:08.059998 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:08.060731 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:08.060742 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:08.061297 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:08.061911 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:08.061930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.063029 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.063033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:08.064475 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:08.064489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.065305 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.065313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:08.065551 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:08.066242 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:08.066267 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:08.068869 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:08.072266 (+ 3397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:08.072278 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:08.072299 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:08.073001 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:08.073534 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:08.073538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:08.073542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:08.073816 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:08.074362 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:08.074371 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:08.074941 (+ 570us) client_negotiation.cc:769] Sending connection context
0806 08:58:08.075269 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 08:58:08.075896 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":355,"threads_started":1}
I20260806 08:58:08.079360 2507 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35903
I20260806 08:58:08.079762 2507 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:08.080642 2507 heartbeater.cc:511] Master 127.22.98.62:35903 requested a full tablet report, sending...
I20260806 08:58:08.082898 2249 ts_manager.cc:194] Registered new tserver with Master: 657401d72d994c7d8965989031fd5cfa (127.22.98.1:39667)
I20260806 08:58:08.084475 2249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:48729
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:58:08.124373 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:08.501500 2514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:08.501825 2514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:08.501925 2514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:08.509387 2514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:08.509510 2514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:08.509658 2514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:58:08.519851 2514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:08.521178 2514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:08.522467 2514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:08.535867 2521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:08.535831 2522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:08.535907 2524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:08.537173 2514 server_base.cc:1034] running on GCE node
I20260806 08:58:08.537880 2514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:08.538973 2514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:08.540179 2514 hybrid_clock.cc:648] HybridClock initialized: now 1786006688540126 us; error 35 us; skew 500 ppm
I20260806 08:58:08.540493 2514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:08 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006688, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:08.543880 2514 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:58:08.545385 2514 webserver.cc:460] Webserver started at http://127.22.98.2:43349/ using document root <none> and password file <none>
I20260806 08:58:08.546085 2514 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:08.546206 2514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:08.546506 2514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:08.549535 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "2ae78eb9fc944ad299d3a8d6fa85a301"
format_stamp: "Formatted at 2026-08-06 08:58:08 on dist-test-slave-jpj1"
server_key: "1504f7edbff8afff8999773a4f38c272"
server_key_iv: "7780bdd1eca9d7b146ff99029be32cc0"
server_key_version: "encryptionkey@0"
I20260806 08:58:08.550369 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "2ae78eb9fc944ad299d3a8d6fa85a301"
format_stamp: "Formatted at 2026-08-06 08:58:08 on dist-test-slave-jpj1"
server_key: "1504f7edbff8afff8999773a4f38c272"
server_key_iv: "7780bdd1eca9d7b146ff99029be32cc0"
server_key_version: "encryptionkey@0"
I20260806 08:58:08.556082 2514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:58:08.559785 2532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:08.561169 2514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:58:08.561336 2514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "2ae78eb9fc944ad299d3a8d6fa85a301"
format_stamp: "Formatted at 2026-08-06 08:58:08 on dist-test-slave-jpj1"
server_key: "1504f7edbff8afff8999773a4f38c272"
server_key_iv: "7780bdd1eca9d7b146ff99029be32cc0"
server_key_version: "encryptionkey@0"
I20260806 08:58:08.561529 2514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:08.582723 2514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:08.587214 2514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:08.587522 2514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:08.588716 2514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:08.590272 2514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:08.590358 2514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:08.590490 2514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:08.590552 2514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:08.627480 2514 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:36789
I20260806 08:58:08.627521 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:36789 every 8 connection(s)
I20260806 08:58:08.628881 2514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:58:08.633795 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2514
I20260806 08:58:08.634114 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:58:08.634581 22920 external_mini_cluster.cc:1428] Setting key 3f2eddc795d285d5a3b35d106512e858
Aug 06 08:58:08 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006688, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:08.651108 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:08.631423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:54895 (local address 127.22.98.62:35903)
0806 08:58:08.631921 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:08.631928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:08.633030 (+ 1102us) server_negotiation.cc:407] Connection header received
0806 08:58:08.634966 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:08.634972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:08.635065 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:08.635204 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:08.637130 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.638163 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.639901 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.640536 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.640656 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:08.644452 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:08.644459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:08.644463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:08.644502 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:08.647763 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:08.648717 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:08.648721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:08.648723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:08.648853 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:08.649340 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:08.649343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:08.649346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:08.649662 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:08.649786 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:08.650656 (+ 870us) server_negotiation.cc:299] Negotiation successful
0806 08:58:08.650841 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":224,"threads_started":1}
I20260806 08:58:08.652657 2675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:08.631994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:54895)
0806 08:58:08.632858 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:08.632900 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:08.634646 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:08.635506 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:08.635516 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:08.636115 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:08.636926 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:08.636950 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.638340 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.638344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:08.639738 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:08.639753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:08.640750 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:08.640755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:08.640977 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:08.641625 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:08.641654 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:08.644180 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:08.647933 (+ 3753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:08.647943 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:08.647965 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:08.648600 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:08.648956 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:08.648962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:08.648965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:08.649229 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:08.649795 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:08.649803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:08.650262 (+ 459us) client_negotiation.cc:769] Sending connection context
0806 08:58:08.650574 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 08:58:08.651151 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":392,"threads_started":1}
I20260806 08:58:08.654227 2672 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35903
I20260806 08:58:08.654556 2672 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:08.655295 2672 heartbeater.cc:511] Master 127.22.98.62:35903 requested a full tablet report, sending...
I20260806 08:58:08.656878 2249 ts_manager.cc:194] Registered new tserver with Master: 2ae78eb9fc944ad299d3a8d6fa85a301 (127.22.98.2:36789)
I20260806 08:58:08.657720 2249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:54895
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:58:08.696249 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:09.062115 2678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:09.062439 2678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:09.062584 2678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:09.070741 2678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:09.070907 2678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:09.071070 2678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:58:09.082187 2678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:09.083636 2678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:09.085047 2678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 08:58:09.087805 2507 heartbeater.cc:503] Master 127.22.98.62:35903 was elected leader, sending a full tablet report...
W20260806 08:58:09.099463 2685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:09.099437 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:09.099386 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:09.100458 2678 server_base.cc:1034] running on GCE node
I20260806 08:58:09.101075 2678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:09.102056 2678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:09.103261 2678 hybrid_clock.cc:648] HybridClock initialized: now 1786006689103198 us; error 60 us; skew 500 ppm
I20260806 08:58:09.103652 2678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:09 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006689, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:09.106999 2678 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:58:09.108606 2678 webserver.cc:460] Webserver started at http://127.22.98.3:37845/ using document root <none> and password file <none>
I20260806 08:58:09.109310 2678 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:09.109444 2678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:09.109776 2678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:09.113092 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "e4460e9a463b40b9a917013cb1e6544e"
format_stamp: "Formatted at 2026-08-06 08:58:09 on dist-test-slave-jpj1"
server_key: "71b5ed84db65b5fce4f67919ef6202b8"
server_key_iv: "ccebd0926e84c6f5df2008ce9b4c8ef1"
server_key_version: "encryptionkey@0"
I20260806 08:58:09.113957 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "e4460e9a463b40b9a917013cb1e6544e"
format_stamp: "Formatted at 2026-08-06 08:58:09 on dist-test-slave-jpj1"
server_key: "71b5ed84db65b5fce4f67919ef6202b8"
server_key_iv: "ccebd0926e84c6f5df2008ce9b4c8ef1"
server_key_version: "encryptionkey@0"
I20260806 08:58:09.119194 2678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 08:58:09.123046 2698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:09.124362 2678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 08:58:09.124512 2678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "e4460e9a463b40b9a917013cb1e6544e"
format_stamp: "Formatted at 2026-08-06 08:58:09 on dist-test-slave-jpj1"
server_key: "71b5ed84db65b5fce4f67919ef6202b8"
server_key_iv: "ccebd0926e84c6f5df2008ce9b4c8ef1"
server_key_version: "encryptionkey@0"
I20260806 08:58:09.124696 2678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:09.145501 2678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:09.150008 2678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:09.150326 2678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:09.151470 2678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:09.152972 2678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:09.153047 2678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:09.153187 2678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:09.153237 2678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:09.188843 2678 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:37865
I20260806 08:58:09.188889 2839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:37865 every 8 connection(s)
I20260806 08:58:09.190153 2678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:58:09.194196 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2678
I20260806 08:58:09.194379 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:58:09.194866 22920 external_mini_cluster.cc:1428] Setting key 5b9fc7aef14f9fd6cedc5333c5482892
Aug 06 08:58:09 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006689, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:09.211947 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:09.192950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:57125 (local address 127.22.98.62:35903)
0806 08:58:09.193425 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:09.193429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:09.195366 (+ 1937us) server_negotiation.cc:407] Connection header received
0806 08:58:09.196682 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:09.196688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:09.196810 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:09.196924 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:09.198574 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.199365 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.200897 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.201325 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.201510 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:09.205023 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:09.205030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:09.205035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:09.205074 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:09.208165 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.209070 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.209076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.209079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.209285 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.209870 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.209874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.209877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.210294 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:09.210463 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:09.211369 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 08:58:09.211632 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":257,"threads_started":1}
I20260806 08:58:09.213142 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:09.193417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:57125)
0806 08:58:09.194326 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:09.194380 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:09.196393 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:09.197186 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:09.197195 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:09.197722 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:09.198407 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:09.198428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.199483 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.199487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:09.200744 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:09.200758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.201578 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.201584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:09.201789 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:09.202444 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:09.202474 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:09.204776 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:09.208328 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:09.208337 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:09.208353 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:09.208889 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:09.209414 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:09.209419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:09.209422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:09.209720 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:09.210456 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:09.210465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:09.210958 (+ 493us) client_negotiation.cc:769] Sending connection context
0806 08:58:09.211338 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 08:58:09.211916 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":421,"threads_started":1}
I20260806 08:58:09.214542 2841 heartbeater.cc:347] Connected to a master server at 127.22.98.62:35903
I20260806 08:58:09.214884 2841 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:09.215773 2841 heartbeater.cc:511] Master 127.22.98.62:35903 requested a full tablet report, sending...
I20260806 08:58:09.217296 2249 ts_manager.cc:194] Registered new tserver with Master: e4460e9a463b40b9a917013cb1e6544e (127.22.98.3:37865)
I20260806 08:58:09.218143 2249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:57125
I20260806 08:58:09.223928 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
Entry for principal oryx/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab.
WARNING: no policy specified for oryx/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
Entry for principal oryx/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab.
WARNING: no policy specified for oryx/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
Entry for principal oryx/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab.
I20260806 08:58:09.394145 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2330
Aug 06 08:58:09 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:58:09.413211 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:09.395355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.22.98.1:39667)
0806 08:58:09.396064 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:09.396072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:09.396101 (+ 29us) server_negotiation.cc:407] Connection header received
0806 08:58:09.396167 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:09.396173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:09.396413 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:09.396549 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:09.397937 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.398955 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.400336 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.400912 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.401030 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:09.406745 (+ 5715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:09.406761 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:09.406779 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:09.406828 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:09.410208 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.410948 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.410958 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.410962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.411104 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.411500 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.411506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.411509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.411946 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:09.412376 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:09.412564 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 08:58:09.412843 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":397,"threads_started":1}
I20260806 08:58:09.414541 2500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:09.497942 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2330
I20260806 08:58:09.521873 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2514
Aug 06 08:58:09 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:58:09.538811 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:09.522826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.22.98.2:36789)
0806 08:58:09.523392 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:09.523400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:09.523426 (+ 26us) server_negotiation.cc:407] Connection header received
0806 08:58:09.523491 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:09.523497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:09.523750 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:09.523922 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:09.525219 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.526264 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.527704 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.528401 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.528523 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:09.533256 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:09.533269 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:09.533285 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:09.533330 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:09.536320 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.536972 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.536978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.536981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.537106 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.537397 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.537400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.537402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.537742 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:09.537984 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:09.538300 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 08:58:09.538515 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":246,"threads_started":1}
I20260806 08:58:09.540222 2667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:09.622327 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2514
I20260806 08:58:09.642861 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2678
Aug 06 08:58:09 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:58:09.659514 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:09.643909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43474 (local address 127.22.98.3:37865)
0806 08:58:09.644430 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:09.644438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:09.644471 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:58:09.644534 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:09.644541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:09.644785 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:09.644913 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:09.646390 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.647387 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.648767 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:09.649368 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:09.649482 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:09.654078 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:09.654092 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:09.654115 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:09.654153 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:09.656844 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.657485 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.657492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.657495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.657622 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:09.658040 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:09.658044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:09.658046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:09.658418 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:09.658663 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:09.658982 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 08:58:09.659197 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":225,"threads_started":1}
I20260806 08:58:09.661145 2835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:09.744624 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2678
I20260806 08:58:09.766378 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2207
I20260806 08:58:09.767712 2285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:09.871990 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2207
I20260806 08:58:09.895365 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35903
--webserver_interface=127.22.98.62
--webserver_port=35875
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:10.234061 2869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:10.234408 2869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:10.234544 2869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:10.241959 2869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:10.242097 2869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:10.242182 2869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:10.242261 2869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:10.242319 2869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:10.252545 2869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=35875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:10.253968 2869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:10.255336 2869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:10.267134 2877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:10.267174 2879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:10.267294 2876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:10.268316 2869 server_base.cc:1034] running on GCE node
I20260806 08:58:10.269119 2869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:10.270606 2869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:10.271814 2869 hybrid_clock.cc:648] HybridClock initialized: now 1786006690271755 us; error 49 us; skew 500 ppm
I20260806 08:58:10.272156 2869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006690, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:10.275214 2869 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:58:10.276757 2869 webserver.cc:460] Webserver started at http://127.22.98.62:35875/ using document root <none> and password file <none>
I20260806 08:58:10.277420 2869 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:10.277535 2869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:10.283099 2869 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 08:58:10.286090 2890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:10.287508 2869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 08:58:10.287750 2869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "a634f95aa59d4232a5867f68c0b69077"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "cb53356946e6d50a40808cc2b5c9ac91"
server_key_iv: "247e5a7ff74c6354da2fd0bd65f9eb1f"
server_key_version: "encryptionkey@0"
I20260806 08:58:10.288746 2869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:10.303076 2869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:10.307482 2869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:10.307809 2869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:10.326333 2869 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:35903
I20260806 08:58:10.326360 2959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:35903 every 8 connection(s)
I20260806 08:58:10.328022 2869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:58:10.330674 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2869
I20260806 08:58:10.331773 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:39667
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:35903
--webserver_port=34405
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:58:10.337680 2960 sys_catalog.cc:263] Verifying existing consensus state
I20260806 08:58:10.342360 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Bootstrap starting.
I20260806 08:58:10.362594 2960 log.cc:824] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:10.371979 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 08:58:10.372500 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Bootstrap complete.
I20260806 08:58:10.381628 2960 raft_consensus.cc:348] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:10.382479 2960 raft_consensus.cc:729] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a634f95aa59d4232a5867f68c0b69077, State: Initialized, Role: FOLLOWER
I20260806 08:58:10.383113 2960 consensus_queue.cc:260] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:10.383348 2960 raft_consensus.cc:388] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:58:10.383458 2960 raft_consensus.cc:482] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:58:10.383671 2960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 1 FOLLOWER]: Advancing to term 2
I20260806 08:58:10.387676 2960 raft_consensus.cc:504] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:10.388204 2960 leader_election.cc:302] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a634f95aa59d4232a5867f68c0b69077; no voters:
I20260806 08:58:10.388833 2960 leader_election.cc:288] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 08:58:10.388975 2964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 08:58:10.390120 2964 raft_consensus.cc:686] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [term 2 LEADER]: Becoming Leader. State: Replica: a634f95aa59d4232a5867f68c0b69077, State: Running, Role: LEADER
I20260806 08:58:10.390743 2964 consensus_queue.cc:237] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } }
I20260806 08:58:10.391223 2960 sys_catalog.cc:564] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:58:10.393386 2966 sys_catalog.cc:455] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a634f95aa59d4232a5867f68c0b69077. Latest consensus state: current_term: 2 leader_uuid: "a634f95aa59d4232a5867f68c0b69077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } } }
I20260806 08:58:10.393731 2966 sys_catalog.cc:458] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:10.394344 2965 sys_catalog.cc:455] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a634f95aa59d4232a5867f68c0b69077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a634f95aa59d4232a5867f68c0b69077" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 35903 } } }
I20260806 08:58:10.394688 2965 sys_catalog.cc:458] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:10.394770 2969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:58:10.407838 2969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:58:10.412238 2969 catalog_manager.cc:1259] Loaded cluster ID: b656ba3fd44449449fadd59923318857
I20260806 08:58:10.412456 2969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:58:10.416596 2969 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:58:10.418242 2969 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a634f95aa59d4232a5867f68c0b69077: Loaded TSK: 0
I20260806 08:58:10.419894 2969 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 08:58:10.682430 2962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:10.682744 2962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:10.682884 2962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:10.690445 2962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:10.690582 2962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:10.690733 2962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:58:10.700913 2962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.1.keytab
--principal=oryx/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:39667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=34405
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:10.702276 2962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:10.703665 2962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:10.718403 2994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:10.718528 2993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:10.719020 2998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:10.719657 2962 server_base.cc:1034] running on GCE node
I20260806 08:58:10.720434 2962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:10.721639 2962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:10.722888 2962 hybrid_clock.cc:648] HybridClock initialized: now 1786006690722821 us; error 55 us; skew 500 ppm
I20260806 08:58:10.723263 2962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006690, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:10.727407 2962 init.cc:377] Logged in from keytab as oryx/127.22.98.1@KRBTEST.COM (short username oryx)
I20260806 08:58:10.729274 2962 webserver.cc:460] Webserver started at http://127.22.98.1:34405/ using document root <none> and password file <none>
I20260806 08:58:10.730219 2962 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:10.730381 2962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:10.737716 2962 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:58:10.741364 3006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:10.742924 2962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 08:58:10.743099 2962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "657401d72d994c7d8965989031fd5cfa"
format_stamp: "Formatted at 2026-08-06 08:58:07 on dist-test-slave-jpj1"
server_key: "159761b2024ebdafcac9c8a7d70d91ff"
server_key_iv: "29c6d3fbbc25369dca9f8e62abbb1df4"
server_key_version: "encryptionkey@0"
I20260806 08:58:10.744135 2962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:10.767251 2962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:10.771471 2962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:10.771806 2962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:10.772854 2962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:10.774363 2962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:10.774474 2962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:10.774590 2962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:10.774659 2962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:10.805538 2962 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:39667
I20260806 08:58:10.805603 3154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:39667 every 8 connection(s)
I20260806 08:58:10.807063 2962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:58:10.811460 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 2962
I20260806 08:58:10.812247 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:36789
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:35903
--webserver_port=43349
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:10.828298 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:10.810373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:41611)
0806 08:58:10.812071 (+ 1698us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:10.812132 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:10.814444 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:10.815992 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:10.816003 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:10.816716 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:10.817523 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.817550 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.819291 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.819296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.821150 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.821167 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.822227 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.822234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.822506 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.823231 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:10.823260 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:10.827219 (+ 3959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1377,"thread_start_us":1169,"threads_started":1}
W20260806 08:58:10.828960 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:10.810307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:41611 (local address 127.22.98.62:35903)
0806 08:58:10.811140 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:10.811157 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:10.812288 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 08:58:10.815057 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:10.815073 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:10.815414 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:10.815817 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:10.817727 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.819113 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.821307 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.821978 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.822123 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.827779 (+ 5656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:41611: BlockingRecv error: recv got EOF from 127.22.98.1:41611 (error 108)
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":275,"threads_started":1}
W20260806 08:58:10.829246 3157 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:35903 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.98.62:35903: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:10.838258 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:10.829853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:49449)
0806 08:58:10.830045 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:10.830065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:10.830172 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:10.830469 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:10.830474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:10.830893 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:10.831106 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.831119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.832203 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.832208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.833437 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.833452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.834116 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.834122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.834300 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.834870 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:10.834891 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:10.837989 (+ 3098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 08:58:10.838428 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:10.830054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:49449 (local address 127.22.98.62:35903)
0806 08:58:10.830202 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:10.830206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:10.830222 (+ 16us) server_negotiation.cc:407] Connection header received
0806 08:58:10.830280 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:10.830284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:10.830340 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:10.830416 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:10.831246 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.832051 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.833579 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.833963 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.834054 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.838214 (+ 4160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:49449: BlockingRecv error: recv got EOF from 127.22.98.1:49449 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:10 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:10.847200 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:10.838963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:60601)
0806 08:58:10.839202 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:10.839221 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:10.839318 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:10.839791 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:10.839794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:10.840058 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:10.840199 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.840206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.841601 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.841604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.842491 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:10.842502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.843395 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.843399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:10.843518 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.844083 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:10.844100 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:10.846886 (+ 2786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260806 08:58:10.847325 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:10.839240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:60601 (local address 127.22.98.62:35903)
0806 08:58:10.839435 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:10.839441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:10.839461 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:58:10.839515 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:10.839520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:10.839642 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:10.839750 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:10.840330 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.841457 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.842628 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:10.843269 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:10.843396 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:10.847074 (+ 3678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:60601: BlockingRecv error: recv got EOF from 127.22.98.1:60601 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 08:58:10.847651 3157 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 08:58:11.161742 3162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:11.162068 3162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:11.162225 3162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:11.169561 3162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:11.169719 3162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:11.169876 3162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:58:11.179921 3162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.2.keytab
--principal=oryx/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:36789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=43349
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:11.181250 3162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:11.182607 3162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:11.196007 3169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:11.196039 3171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:11.196106 3168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:11.196887 3162 server_base.cc:1034] running on GCE node
I20260806 08:58:11.197547 3162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:11.198508 3162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:11.199733 3162 hybrid_clock.cc:648] HybridClock initialized: now 1786006691199669 us; error 53 us; skew 500 ppm
I20260806 08:58:11.200078 3162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006691, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:11.203198 3162 init.cc:377] Logged in from keytab as oryx/127.22.98.2@KRBTEST.COM (short username oryx)
I20260806 08:58:11.204869 3162 webserver.cc:460] Webserver started at http://127.22.98.2:43349/ using document root <none> and password file <none>
I20260806 08:58:11.205590 3162 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:11.205734 3162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:11.211459 3162 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 08:58:11.214339 3179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.215793 3162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:58:11.215948 3162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "2ae78eb9fc944ad299d3a8d6fa85a301"
format_stamp: "Formatted at 2026-08-06 08:58:08 on dist-test-slave-jpj1"
server_key: "1504f7edbff8afff8999773a4f38c272"
server_key_iv: "7780bdd1eca9d7b146ff99029be32cc0"
server_key_version: "encryptionkey@0"
I20260806 08:58:11.216899 3162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:11.237593 3162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:11.241909 3162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:11.242157 3162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:11.243125 3162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:11.244552 3162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:11.244668 3162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.244750 3162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:11.244805 3162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.276705 3162 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:36789
I20260806 08:58:11.276789 3332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:36789 every 8 connection(s)
I20260806 08:58:11.278178 3162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:58:11.285308 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 3162
I20260806 08:58:11.285873 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:37865
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:35903
--webserver_port=37845
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 08:58:11.293720 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.280669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:38161 (local address 127.22.98.62:35903)
0806 08:58:11.280897 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.280904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.282105 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 08:58:11.283921 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.283926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.283993 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.284079 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.285576 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.286375 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.287937 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.288347 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.288462 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.293451 (+ 4989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:38161: BlockingRecv error: recv got EOF from 127.22.98.2:38161 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 08:58:11.294421 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.281204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:38161)
0806 08:58:11.281939 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.281986 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.283657 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.284279 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.284287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.284800 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.285377 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.285401 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.286539 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.286543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.287775 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.287788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.288583 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.288592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.288812 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.289452 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.289480 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.293227 (+ 3747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":283,"threads_started":1}
W20260806 08:58:11.295089 3333 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:35903 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.98.62:35903: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:11.304008 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.295677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:60781)
0806 08:58:11.295975 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.295991 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.296073 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.296386 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.296389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.296678 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.296832 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.296840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.297978 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.297983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.299239 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.299255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.299964 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.299969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.300129 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.300656 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.300676 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.303745 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183}
W20260806 08:58:11.304142 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.295884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:60781 (local address 127.22.98.62:35903)
0806 08:58:11.296060 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.296066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.296087 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:58:11.296192 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.296197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.296271 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.296373 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.296969 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.297835 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.299362 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.299815 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.299916 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.303945 (+ 4029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:60781: BlockingRecv error: recv got EOF from 127.22.98.2:60781 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:11.312757 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.304763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:58309)
0806 08:58:11.304996 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.305011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.305088 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.305357 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.305360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.305615 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.305759 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.305768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.306942 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.306945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.307886 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.307899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.308822 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.308828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.308981 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.309547 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.309564 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.312537 (+ 2973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260806 08:58:11.313050 3333 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 08:58:11.313244 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.304902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:58309 (local address 127.22.98.62:35903)
0806 08:58:11.305080 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.305085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.305103 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:58:11.305175 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.305179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.305243 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.305324 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.305912 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.306826 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.308019 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.308592 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.308715 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.312877 (+ 4162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:58309: BlockingRecv error: recv got EOF from 127.22.98.2:58309 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 08:58:11.629483 3336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:11.629802 3336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:11.629933 3336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:11.637070 3336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:11.637193 3336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:11.637348 3336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:58:11.647100 3336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/krb5kdc/oryx_127.22.98.3.keytab
--principal=oryx/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:37865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=37845
--tserver_master_addrs=127.22.98.62:35903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:11.648526 3336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:11.649976 3336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:11.663692 3342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:11.663754 3343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:11.664374 3336 server_base.cc:1034] running on GCE node
W20260806 08:58:11.664458 3347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:11.665278 3336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:11.666318 3336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:11.667518 3336 hybrid_clock.cc:648] HybridClock initialized: now 1786006691667450 us; error 68 us; skew 500 ppm
I20260806 08:58:11.667865 3336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006691, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:11.671061 3336 init.cc:377] Logged in from keytab as oryx/127.22.98.3@KRBTEST.COM (short username oryx)
I20260806 08:58:11.672590 3336 webserver.cc:460] Webserver started at http://127.22.98.3:37845/ using document root <none> and password file <none>
I20260806 08:58:11.673295 3336 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:11.673431 3336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:11.679044 3336 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 08:58:11.681921 3354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.683276 3336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 08:58:11.683439 3336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "e4460e9a463b40b9a917013cb1e6544e"
format_stamp: "Formatted at 2026-08-06 08:58:09 on dist-test-slave-jpj1"
server_key: "71b5ed84db65b5fce4f67919ef6202b8"
server_key_iv: "ccebd0926e84c6f5df2008ce9b4c8ef1"
server_key_version: "encryptionkey@0"
I20260806 08:58:11.684458 3336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:11.708952 3336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:11.713438 3336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:11.713740 3336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:11.714751 3336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:11.716212 3336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:11.716315 3336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.716399 3336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:11.716454 3336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:11.752794 3336 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:37865
I20260806 08:58:11.752875 3502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:37865 every 8 connection(s)
I20260806 08:58:11.754292 3336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:58:11.757166 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 3336
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 08:58:11.773142 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.756946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:50017 (local address 127.22.98.62:35903)
0806 08:58:11.757169 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.757175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.758475 (+ 1300us) server_negotiation.cc:407] Connection header received
0806 08:58:11.761368 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.761372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.761443 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.761526 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.763729 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.764708 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.766837 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.767460 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.767571 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.772849 (+ 5278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:50017: BlockingRecv error: recv got EOF from 127.22.98.3:50017 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 08:58:11.773639 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.757440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:50017)
0806 08:58:11.758252 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.758349 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.761027 (+ 2678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.761818 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.761831 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.762633 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.763500 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.763526 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.764886 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.764892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.766690 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.766703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.767708 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.767716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.768022 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.768749 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.768780 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.772569 (+ 3789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":308,"threads_started":1}
W20260806 08:58:11.774401 3503 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:35903 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.98.62:35903: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
I20260806 08:58:11.779735 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.758169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.22.98.62:35903)
0806 08:58:11.763948 (+ 5779us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.763954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.763970 (+ 16us) server_negotiation.cc:407] Connection header received
0806 08:58:11.764019 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.764024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.764089 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.764201 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.766284 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.767212 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.768877 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.769308 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.769398 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.772294 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:11.772304 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:11.772323 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:11.772365 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:11.776015 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:11.776838 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:11.776846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:11.776852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:11.777091 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:11.777547 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:11.777553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:11.777556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:11.778166 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:11.778476 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:11.778952 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 08:58:11.779358 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5661,"thread_start_us":5542,"threads_started":1}
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:11.783737 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.774930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:56969)
0806 08:58:11.775125 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.775146 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.775249 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.775560 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.775566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.775969 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.776222 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.776235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.777476 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.777479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.778904 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.778920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.779792 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.779798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.780022 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.780551 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.780568 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.783492 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 08:58:11.784108 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.775070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:56969 (local address 127.22.98.62:35903)
0806 08:58:11.775233 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.775237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.775252 (+ 15us) server_negotiation.cc:407] Connection header received
0806 08:58:11.775353 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.775357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.775412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.775485 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.776377 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.777296 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.779074 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.779629 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.779758 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.783759 (+ 4001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:56969: BlockingRecv error: recv got EOF from 127.22.98.3:56969 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:11.792752 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.784471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:38673)
0806 08:58:11.784612 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.784628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.784705 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.785142 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.785147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.785558 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.785806 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.785816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.786944 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.786949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.788022 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.788034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.788869 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.788874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.789006 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.789531 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.789547 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.792508 (+ 2961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 08:58:11.792965 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.784669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:38673 (local address 127.22.98.62:35903)
0806 08:58:11.784906 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.784909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.784923 (+ 14us) server_negotiation.cc:407] Connection header received
0806 08:58:11.784961 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.784964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.785016 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.785097 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.785947 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.786802 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.788157 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.788685 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.788821 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.792723 (+ 3902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:38673: BlockingRecv error: recv got EOF from 127.22.98.3:38673 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
W20260806 08:58:11.793107 3503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:11 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:11.856950 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:11.848270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:48645)
0806 08:58:11.848797 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:11.848814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:11.848933 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:11.849201 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:11.849204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:11.849681 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:11.849916 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.849927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.851002 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.851005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.851988 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:11.852000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.852710 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.852715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:11.852864 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.853487 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:11.853506 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:11.856650 (+ 3144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":265,"threads_started":1}
W20260806 08:58:11.857092 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:11.848337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:48645 (local address 127.22.98.62:35903)
0806 08:58:11.848562 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:11.848569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:11.848893 (+ 324us) server_negotiation.cc:407] Connection header received
0806 08:58:11.849028 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:11.849032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:11.849094 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:11.849175 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:11.850046 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.850794 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.852108 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:11.852550 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:11.852648 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:11.856879 (+ 4231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:48645: BlockingRecv error: recv got EOF from 127.22.98.1:48645 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:12.326326 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:12.313711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:34521)
0806 08:58:12.314384 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:12.314403 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:12.314533 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:12.314858 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:12.314862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:12.315341 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:12.315580 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.315593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.316641 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.316646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.317897 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.317910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.318760 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.318768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.318920 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.322929 (+ 4009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:12.322951 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:12.326081 (+ 3130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":366,"threads_started":1}
W20260806 08:58:12.326817 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:12.313767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:34521 (local address 127.22.98.62:35903)
0806 08:58:12.313981 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:12.313989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:12.314496 (+ 507us) server_negotiation.cc:407] Connection header received
0806 08:58:12.314641 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:12.314645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:12.314710 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:12.314802 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:12.315722 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.316495 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.318057 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.318573 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.318683 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.326364 (+ 7681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:34521: BlockingRecv error: recv got EOF from 127.22.98.2:34521 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:12.802744 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:12.793739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:56621)
0806 08:58:12.794403 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:12.794425 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:12.794546 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:12.794833 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:12.794836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:12.795318 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:12.795517 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.795526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.796720 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.796725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.797877 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.797894 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.798754 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.798761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.798946 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.799586 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:12.799611 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:12.802505 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":408,"threads_started":1}
W20260806 08:58:12.803027 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:12.793795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:56621 (local address 127.22.98.62:35903)
0806 08:58:12.793998 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:12.794003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:12.794493 (+ 490us) server_negotiation.cc:407] Connection header received
0806 08:58:12.794644 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:12.794648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:12.794711 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:12.794795 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:12.795722 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.796517 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.798030 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.798571 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.798701 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.802796 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:56621: BlockingRecv error: recv got EOF from 127.22.98.3:56621 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:12 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:12.866937 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:12.858045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:53403)
0806 08:58:12.858627 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:12.858647 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:12.858791 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:12.859174 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:12.859179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:12.859757 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:12.859951 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.859962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.860994 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.860999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.862062 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:12.862075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.862778 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.862784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:12.862953 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.863677 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:12.863696 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:12.866579 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":306,"threads_started":1}
W20260806 08:58:12.867079 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:12.858107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:53403 (local address 127.22.98.62:35903)
0806 08:58:12.858312 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:12.858318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:12.858743 (+ 425us) server_negotiation.cc:407] Connection header received
0806 08:58:12.858907 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:12.858913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:12.859021 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:12.859115 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:12.860094 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.860848 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.862201 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:12.862616 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:12.862740 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:12.866817 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:53403: BlockingRecv error: recv got EOF from 127.22.98.1:53403 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:13.336617 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:13.327221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:45133)
0806 08:58:13.327795 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:13.327816 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:13.327965 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:13.328262 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:13.328267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:13.328819 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:13.329103 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.329119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.330229 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.330236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.331739 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.331765 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.332493 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.332499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.332669 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.333293 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:13.333317 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:13.336321 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":312,"threads_started":1}
W20260806 08:58:13.336943 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:13.327305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:45133 (local address 127.22.98.62:35903)
0806 08:58:13.327518 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:13.327526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:13.327903 (+ 377us) server_negotiation.cc:407] Connection header received
0806 08:58:13.328069 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:13.328073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:13.328141 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:13.328222 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:13.329239 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.330081 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.331884 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.332333 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.332489 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.336723 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:45133: BlockingRecv error: recv got EOF from 127.22.98.2:45133 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:13.813077 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:13.803713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:44567)
0806 08:58:13.804211 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:13.804233 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:13.804364 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:13.804669 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:13.804672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:13.805089 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:13.805243 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.805253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.806744 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.806749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.807713 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.807728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.808646 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.808653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.808827 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.809429 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:13.809455 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:13.812747 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":268,"threads_started":1}
W20260806 08:58:13.813266 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:13.803851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:44567 (local address 127.22.98.62:35903)
0806 08:58:13.804074 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:13.804080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:13.804307 (+ 227us) server_negotiation.cc:407] Connection header received
0806 08:58:13.804468 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:13.804472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:13.804536 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:13.804614 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:13.805418 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.806577 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.807888 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.808458 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.808559 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.813004 (+ 4445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:44567: BlockingRecv error: recv got EOF from 127.22.98.3:44567 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:13 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:13.876237 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:13.867955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:34951)
0806 08:58:13.868506 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:13.868526 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:13.868646 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:13.869043 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:13.869048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:13.869522 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:13.869724 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.869734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.870756 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.870761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.871746 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:13.871759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.872358 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.872362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:13.872484 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.873119 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:13.873161 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:13.875978 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":291,"threads_started":1}
W20260806 08:58:13.876391 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:13.868042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:34951 (local address 127.22.98.62:35903)
0806 08:58:13.868281 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:13.868288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:13.868611 (+ 323us) server_negotiation.cc:407] Connection header received
0806 08:58:13.868747 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:13.868751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:13.868865 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:13.868985 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:13.869874 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.870627 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.871862 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:13.872245 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:13.872354 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:13.876164 (+ 3810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:34951: BlockingRecv error: recv got EOF from 127.22.98.1:34951 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:14.346263 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:14.337642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:39769)
0806 08:58:14.338315 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:14.338331 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:14.338467 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:14.338802 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:14.338807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:14.339169 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:14.339355 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.339365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.340474 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.340478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.341385 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.341398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.342078 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.342084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.342259 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.342839 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:14.342857 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:14.345920 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":434,"threads_started":1}
W20260806 08:58:14.346400 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:14.337804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:39769 (local address 127.22.98.62:35903)
0806 08:58:14.338056 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:14.338063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:14.338414 (+ 351us) server_negotiation.cc:407] Connection header received
0806 08:58:14.338593 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:14.338597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:14.338662 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:14.338741 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:14.339503 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.340329 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.341525 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.341931 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.342082 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.346154 (+ 4072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:39769: BlockingRecv error: recv got EOF from 127.22.98.2:39769 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:14.823184 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:14.814207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:60453)
0806 08:58:14.814678 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:14.814702 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:14.814828 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:14.815109 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:14.815112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:14.815472 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:14.815829 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.815844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.817094 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.817098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.818192 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.818207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.818889 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.818895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.819071 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.819683 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:14.819700 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:14.822840 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
W20260806 08:58:14.823309 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:14.814345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:60453 (local address 127.22.98.62:35903)
0806 08:58:14.814587 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:14.814594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:14.814779 (+ 185us) server_negotiation.cc:407] Connection header received
0806 08:58:14.814928 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:14.814932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:14.814998 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:14.815080 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:14.815990 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.816940 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.818327 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.818749 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.818859 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.823074 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:60453: BlockingRecv error: recv got EOF from 127.22.98.3:60453 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:14 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:14.887046 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:14.877205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:40135)
0806 08:58:14.877725 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:14.877744 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:14.877892 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:14.878230 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:14.878234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:14.878646 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:14.878815 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.878826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.879959 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.879965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.880937 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:14.880951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.881688 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.881694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:14.881872 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.882449 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:14.882468 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:14.886702 (+ 4234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":266,"threads_started":1}
W20260806 08:58:14.889516 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:14.877254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:40135 (local address 127.22.98.62:35903)
0806 08:58:14.877536 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:14.877540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:14.877853 (+ 313us) server_negotiation.cc:407] Connection header received
0806 08:58:14.878023 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:14.878029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:14.878098 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:14.878176 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:14.878964 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.879834 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.881085 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:14.881523 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:14.881633 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:14.886989 (+ 5356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:40135: BlockingRecv error: recv got EOF from 127.22.98.1:40135 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:15.356140 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:15.347298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:45949)
0806 08:58:15.347848 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:15.347866 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:15.347987 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:15.348263 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:15.348266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:15.348763 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:15.348932 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.348942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.350205 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.350210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.351179 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.351191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.351929 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.351934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.352081 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.352645 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:15.352664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:15.355840 (+ 3176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":309,"threads_started":1}
W20260806 08:58:15.356233 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:15.347402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:45949 (local address 127.22.98.62:35903)
0806 08:58:15.347594 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:15.347599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:15.347965 (+ 366us) server_negotiation.cc:407] Connection header received
0806 08:58:15.348078 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:15.348081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:15.348147 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:15.348233 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:15.349102 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.350070 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.351338 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.351798 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.351897 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.356047 (+ 4150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:45949: BlockingRecv error: recv got EOF from 127.22.98.2:45949 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:15.832751 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:15.824186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:54391)
0806 08:58:15.824701 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:15.824723 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:15.824885 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:15.825306 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:15.825309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:15.825844 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:15.826054 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.826068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.827108 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.827112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.828112 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.828124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.828798 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.828802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.828939 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.829554 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:15.829582 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:15.832422 (+ 2840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":286,"threads_started":1}
W20260806 08:58:15.832887 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:15.824274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:54391 (local address 127.22.98.62:35903)
0806 08:58:15.824479 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:15.824486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:15.824812 (+ 326us) server_negotiation.cc:407] Connection header received
0806 08:58:15.825030 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:15.825035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:15.825149 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:15.825260 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:15.826209 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.826995 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.828235 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.828645 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.828738 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.832650 (+ 3912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:54391: BlockingRecv error: recv got EOF from 127.22.98.3:54391 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:15 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:15.896368 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:15.888124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:41305)
0806 08:58:15.888673 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:15.888690 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:15.888809 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:15.889100 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:15.889103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:15.889553 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:15.889742 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.889752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.890694 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.890697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.891656 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:15.891670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.892415 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.892420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:15.892558 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.893198 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:15.893217 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:15.896104 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":281,"threads_started":1}
W20260806 08:58:15.896857 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:15.888173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:41305 (local address 127.22.98.62:35903)
0806 08:58:15.888431 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:15.888438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:15.888769 (+ 331us) server_negotiation.cc:407] Connection header received
0806 08:58:15.888921 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:15.888925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:15.888989 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:15.889066 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:15.889878 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.890583 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.891822 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:15.892281 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:15.892384 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:15.896361 (+ 3977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:41305: BlockingRecv error: recv got EOF from 127.22.98.1:41305 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:16.365607 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:16.357213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:35023)
0806 08:58:16.358009 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:16.358026 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:16.358155 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:16.358496 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:16.358501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:16.358911 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:16.359079 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.359089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.360202 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.360208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.361125 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.361138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.361813 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.361820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.361994 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.362571 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:16.362590 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:16.365353 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":314,"threads_started":1}
W20260806 08:58:16.365805 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:16.357396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:35023 (local address 127.22.98.62:35903)
0806 08:58:16.357599 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:16.357607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:16.358105 (+ 498us) server_negotiation.cc:407] Connection header received
0806 08:58:16.358291 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:16.358297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:16.358362 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:16.358439 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:16.359222 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.360064 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.361263 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.361673 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.361795 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.365542 (+ 3747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:35023: BlockingRecv error: recv got EOF from 127.22.98.2:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:16.841699 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:16.833819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:59925)
0806 08:58:16.834348 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:16.834369 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:16.834483 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:16.834747 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:16.834750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:16.835217 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:16.835413 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.835423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.836362 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.836366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.837298 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.837311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.837947 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.837951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.838074 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.838633 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:16.838650 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:16.841445 (+ 2795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":242,"threads_started":1}
W20260806 08:58:16.841843 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:16.833880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59925 (local address 127.22.98.62:35903)
0806 08:58:16.834048 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:16.834052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:16.834434 (+ 382us) server_negotiation.cc:407] Connection header received
0806 08:58:16.834572 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:16.834576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:16.834640 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:16.834717 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:16.835526 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.836241 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.837426 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.837832 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.837924 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.841636 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:59925: BlockingRecv error: recv got EOF from 127.22.98.3:59925 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:16 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:16.906646 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:16.897360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:36757)
0806 08:58:16.897898 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:16.897915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:16.898041 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:16.898380 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:16.898383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:16.899051 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:16.899224 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.899235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.900332 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.900336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.901315 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:16.901328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.902308 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.902324 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:16.902508 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.903246 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:16.903266 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:16.906204 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":290,"threads_started":1}
W20260806 08:58:16.906692 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:16.897427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:36757 (local address 127.22.98.62:35903)
0806 08:58:16.897706 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:16.897711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:16.897991 (+ 280us) server_negotiation.cc:407] Connection header received
0806 08:58:16.898179 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:16.898184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:16.898252 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:16.898344 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:16.899382 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.900197 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.901463 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:16.902114 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:16.902240 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:16.906440 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:36757: BlockingRecv error: recv got EOF from 127.22.98.1:36757 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:17.379052 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:17.366570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:37727)
0806 08:58:17.367325 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:17.367344 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:17.367464 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:17.367841 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:17.367846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:17.368390 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:17.368596 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.368618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.369749 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.369754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.370682 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.370694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.371402 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.371407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.371551 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.375931 (+ 4380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:17.375951 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:17.378822 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":515,"threads_started":1}
W20260806 08:58:17.379360 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:17.366691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:37727 (local address 127.22.98.62:35903)
0806 08:58:17.366902 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:17.366908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:17.367428 (+ 520us) server_negotiation.cc:407] Connection header received
0806 08:58:17.367593 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:17.367598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:17.367673 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:17.367776 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:17.368816 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.369590 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.370817 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.371244 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.371344 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.379094 (+ 7750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:37727: BlockingRecv error: recv got EOF from 127.22.98.2:37727 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:17.851830 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:17.842893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:41413)
0806 08:58:17.843394 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:17.843416 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:17.843566 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:17.843875 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:17.843878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:17.844275 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:17.844448 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.844456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.845531 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.845535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.846556 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.846570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.847302 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.847306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.847429 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.848184 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:17.848210 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:17.851569 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":276,"threads_started":1}
W20260806 08:58:17.852025 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:17.843008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:41413 (local address 127.22.98.62:35903)
0806 08:58:17.843272 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:17.843278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:17.843498 (+ 220us) server_negotiation.cc:407] Connection header received
0806 08:58:17.843680 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:17.843684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:17.843753 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:17.843842 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:17.844561 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.845339 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.846715 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.847165 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.847263 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.851803 (+ 4540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:41413: BlockingRecv error: recv got EOF from 127.22.98.3:41413 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:17 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:17.916172 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:17.907687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:35707)
0806 08:58:17.908302 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:17.908319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:17.908438 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:17.908707 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:17.908710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:17.909323 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:17.909537 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.909548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.910821 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.910825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.911770 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:17.911782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.912432 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.912437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:17.912620 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.913143 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:17.913161 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:17.915910 (+ 2749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":303,"threads_started":1}
W20260806 08:58:17.916299 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:17.907786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:35707 (local address 127.22.98.62:35903)
0806 08:58:17.908043 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:17.908050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:17.908396 (+ 346us) server_negotiation.cc:407] Connection header received
0806 08:58:17.908532 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:17.908536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:17.908599 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:17.908678 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:17.909704 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.910674 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.911901 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:17.912294 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:17.912394 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:17.916106 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:35707: BlockingRecv error: recv got EOF from 127.22.98.1:35707 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:18.388489 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:18.380079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:32845)
0806 08:58:18.380707 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:18.380723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:18.380838 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:18.381103 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:18.381106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:18.381510 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:18.381707 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.381717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.382719 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.382723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.383667 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.383680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.384358 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.384363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.384476 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.385064 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:18.385083 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:18.388137 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":254,"threads_started":1}
W20260806 08:58:18.388651 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:18.380211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:32845 (local address 127.22.98.62:35903)
0806 08:58:18.380407 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:18.380412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:18.380791 (+ 379us) server_negotiation.cc:407] Connection header received
0806 08:58:18.380922 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:18.380927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:18.380993 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:18.381070 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:18.381840 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.382598 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.383798 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.384223 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.384320 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.388389 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:32845: BlockingRecv error: recv got EOF from 127.22.98.2:32845 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:18.861541 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:18.852752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:49785)
0806 08:58:18.853299 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:18.853338 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:18.853457 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:18.853733 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:18.853737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:18.854286 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:18.854552 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.854564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.855676 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.855683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.856688 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.856702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.857441 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.857446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.857561 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.858137 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:18.858156 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:18.861277 (+ 3121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":241,"threads_started":1}
W20260806 08:58:18.861748 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:18.852821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:49785 (local address 127.22.98.62:35903)
0806 08:58:18.853031 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:18.853038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:18.853418 (+ 380us) server_negotiation.cc:407] Connection header received
0806 08:58:18.853545 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:18.853549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:18.853614 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:18.853691 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:18.854678 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.855503 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.856807 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.857299 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.857397 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.861512 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:49785: BlockingRecv error: recv got EOF from 127.22.98.3:49785 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:18 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:18.926332 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:18.917227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:38929)
0806 08:58:18.917789 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:18.917805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:18.917942 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:18.918303 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:18.918308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:18.918742 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:18.919016 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.919028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.920366 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.920372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.921550 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:18.921565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.922359 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.922367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:18.922546 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.923148 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:18.923166 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:18.926104 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":264,"threads_started":1}
W20260806 08:58:18.926537 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:18.917466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:38929 (local address 127.22.98.62:35903)
0806 08:58:18.917694 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:18.917700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:18.917883 (+ 183us) server_negotiation.cc:407] Connection header received
0806 08:58:18.918080 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:18.918084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:18.918147 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:18.918231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:18.919172 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.920216 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.921688 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:18.922117 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:18.922232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:18.926302 (+ 4070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:38929: BlockingRecv error: recv got EOF from 127.22.98.1:38929 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:19.398602 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:19.389513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:33651)
0806 08:58:19.390129 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:19.390145 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:19.390267 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:19.390538 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:19.390541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:19.390979 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:19.391133 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.391142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.392550 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.392554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.393520 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.393535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.394480 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.394485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.394605 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.395208 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:19.395226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:19.398367 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":347,"threads_started":1}
W20260806 08:58:19.398797 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:19.389632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:33651 (local address 127.22.98.62:35903)
0806 08:58:19.389846 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:19.389851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:19.390224 (+ 373us) server_negotiation.cc:407] Connection header received
0806 08:58:19.390354 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:19.390358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:19.390421 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:19.390510 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:19.391264 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.392364 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.393643 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.394318 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.394417 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.398577 (+ 4160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:33651: BlockingRecv error: recv got EOF from 127.22.98.2:33651 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:19.870982 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:19.862466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:48593)
0806 08:58:19.863001 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:19.863022 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:19.863137 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:19.863499 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:19.863502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:19.863978 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:19.864144 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.864154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.865349 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.865354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.866425 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.866439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.867177 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.867182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.867321 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.867928 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:19.867946 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:19.870745 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":281,"threads_started":1}
W20260806 08:58:19.871227 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:19.862567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:48593 (local address 127.22.98.62:35903)
0806 08:58:19.862789 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:19.862794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:19.863102 (+ 308us) server_negotiation.cc:407] Connection header received
0806 08:58:19.863302 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:19.863306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:19.863374 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:19.863459 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:19.864314 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.865179 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.866578 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.867031 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.867134 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.870954 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:48593: BlockingRecv error: recv got EOF from 127.22.98.3:48593 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:19 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:19.935950 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:19.927289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:54623)
0806 08:58:19.927854 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:19.927870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:19.927985 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:19.928249 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:19.928252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:19.928650 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:19.928852 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.928863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.929908 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.929912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.931091 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:19.931105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.931900 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.931905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:19.932031 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.932573 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:19.932591 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:19.935556 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":308,"threads_started":1}
W20260806 08:58:19.936026 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:19.927362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54623 (local address 127.22.98.62:35903)
0806 08:58:19.927564 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:19.927570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:19.927934 (+ 364us) server_negotiation.cc:407] Connection header received
0806 08:58:19.928074 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:19.928078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:19.928140 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:19.928220 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:19.928992 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.929766 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.931228 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:19.931753 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:19.931862 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:19.935804 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:54623: BlockingRecv error: recv got EOF from 127.22.98.1:54623 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:20.408685 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:20.399765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:41487)
0806 08:58:20.400379 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:20.400409 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:20.400557 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:20.400980 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:20.400985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:20.401483 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:20.401666 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.401677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.402700 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.402705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.403849 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.403862 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.404624 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.404629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.404770 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.405342 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:20.405361 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:20.408311 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":340,"threads_started":1}
W20260806 08:58:20.408813 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:20.399945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:41487 (local address 127.22.98.62:35903)
0806 08:58:20.400111 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:20.400117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:20.400510 (+ 393us) server_negotiation.cc:407] Connection header received
0806 08:58:20.400685 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:20.400689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:20.400777 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:20.400875 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:20.401801 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.402571 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.403998 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.404455 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.404557 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.408568 (+ 4011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:41487: BlockingRecv error: recv got EOF from 127.22.98.2:41487 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:20.885723 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:20.874042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:46767)
0806 08:58:20.875959 (+ 1917us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:20.875985 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:20.876130 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:20.876563 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:20.876568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:20.877203 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:20.877451 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.877467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.879060 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.879063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.880569 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.880587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.881337 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.881344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.881496 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.882216 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:20.882241 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:20.885466 (+ 3225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1792,"thread_start_us":1640,"threads_started":1}
W20260806 08:58:20.885874 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:20.874170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:46767 (local address 127.22.98.62:35903)
0806 08:58:20.874357 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:20.874362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:20.876070 (+ 1708us) server_negotiation.cc:407] Connection header received
0806 08:58:20.876271 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:20.876277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:20.876371 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:20.876478 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:20.877658 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.878908 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.880729 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.881147 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.881254 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.885675 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:46767: BlockingRecv error: recv got EOF from 127.22.98.3:46767 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:20 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:20.945562 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:20.936968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:54027)
0806 08:58:20.937488 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:20.937505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:20.937636 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:20.937951 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:20.937957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:20.938373 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:20.938527 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.938537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.939632 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.939636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.940565 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:20.940578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.941307 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.941313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:20.941477 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.942193 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:20.942212 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:20.945332 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":301,"threads_started":1}
W20260806 08:58:20.945814 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:20.937067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54027 (local address 127.22.98.62:35903)
0806 08:58:20.937267 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:20.937273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:20.937586 (+ 313us) server_negotiation.cc:407] Connection header received
0806 08:58:20.937748 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:20.937752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:20.937816 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:20.937903 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:20.938676 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.939462 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.940706 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:20.941153 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:20.941266 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:20.945521 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:54027: BlockingRecv error: recv got EOF from 127.22.98.1:54027 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:21.418915 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:21.409954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:36009)
0806 08:58:21.410468 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:21.410487 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:21.410653 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:21.411035 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:21.411041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:21.411499 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:21.411714 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.411728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.412883 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.412887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.414064 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.414079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.414845 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.414849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.414990 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.415672 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:21.415691 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:21.418673 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
W20260806 08:58:21.419195 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:21.410053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:36009 (local address 127.22.98.62:35903)
0806 08:58:21.410243 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:21.410250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:21.410579 (+ 329us) server_negotiation.cc:407] Connection header received
0806 08:58:21.410791 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:21.410797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:21.410875 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:21.410984 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:21.411897 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.412750 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.414215 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.414675 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.414783 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.418933 (+ 4150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:36009: BlockingRecv error: recv got EOF from 127.22.98.2:36009 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:21.895409 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:21.886661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:46093)
0806 08:58:21.887200 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:21.887222 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:21.887336 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:21.887692 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:21.887696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:21.888091 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:21.888269 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.888282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.889341 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.889347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.890472 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.890486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.891215 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.891220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.891416 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.892092 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:21.892122 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:21.895084 (+ 2962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":303,"threads_started":1}
W20260806 08:58:21.895659 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:21.886866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:46093 (local address 127.22.98.62:35903)
0806 08:58:21.887140 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:21.887163 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:21.887277 (+ 114us) server_negotiation.cc:407] Connection header received
0806 08:58:21.887437 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:21.887441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:21.887506 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:21.887633 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:21.888431 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.889200 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.890614 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.891050 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.891147 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.895382 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:46093: BlockingRecv error: recv got EOF from 127.22.98.3:46093 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:21 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:21.955109 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:21.946439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:37703)
0806 08:58:21.947014 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:21.947030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:21.947164 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:21.947560 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:21.947566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:21.948005 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:21.948197 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.948208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.949251 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.949258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.950234 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:21.950248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.951008 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.951015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:21.951172 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.951822 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:21.951841 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:21.954775 (+ 2934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":308,"threads_started":1}
W20260806 08:58:21.955241 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:21.946560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:37703 (local address 127.22.98.62:35903)
0806 08:58:21.946794 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:21.946801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:21.947110 (+ 309us) server_negotiation.cc:407] Connection header received
0806 08:58:21.947314 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:21.947320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:21.947400 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:21.947485 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:21.948347 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.949097 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.950382 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:21.950854 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:21.950968 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:21.955010 (+ 4042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:37703: BlockingRecv error: recv got EOF from 127.22.98.1:37703 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:22.433651 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:22.420075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:51613)
0806 08:58:22.420838 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:22.420856 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:22.421000 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:22.421411 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:22.421414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:22.422012 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:22.422228 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.422238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.423320 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.423325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.424281 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.424295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.425110 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.425115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.425271 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.429755 (+ 4484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:22.429779 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:22.433322 (+ 3543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":489,"threads_started":1}
W20260806 08:58:22.433938 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:22.420204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:51613 (local address 127.22.98.62:35903)
0806 08:58:22.420390 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:22.420395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:22.420944 (+ 549us) server_negotiation.cc:407] Connection header received
0806 08:58:22.421167 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:22.421172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:22.421264 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:22.421374 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:22.422367 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.423167 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.424461 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.424926 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.425037 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.433635 (+ 8598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:51613: BlockingRecv error: recv got EOF from 127.22.98.2:51613 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:22.906056 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:22.896482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:52829)
0806 08:58:22.897131 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:22.897154 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:22.897284 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:22.897598 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:22.897602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:22.898124 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:22.898430 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.898441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.899590 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.899596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.900537 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.900551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.901372 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.901377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.901514 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.902231 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:22.902250 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:22.905737 (+ 3487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":411,"threads_started":1}
W20260806 08:58:22.906253 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:22.896569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:52829 (local address 127.22.98.62:35903)
0806 08:58:22.896732 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:22.896737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:22.897265 (+ 528us) server_negotiation.cc:407] Connection header received
0806 08:58:22.897383 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:22.897387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:22.897458 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:22.897545 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:22.898589 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.899394 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.900719 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.901184 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.901302 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.906005 (+ 4703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:52829: BlockingRecv error: recv got EOF from 127.22.98.3:52829 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:22 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:22.965258 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:22.956227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:46817)
0806 08:58:22.956951 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:22.956974 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:22.957143 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:22.957577 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:22.957583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:22.958093 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:22.958312 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.958323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.959408 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.959413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.960391 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:22.960405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.961209 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.961214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:22.961359 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.961990 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:22.962009 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:22.965005 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":408,"threads_started":1}
W20260806 08:58:22.965454 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:22.956335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:46817 (local address 127.22.98.62:35903)
0806 08:58:22.956548 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:22.956559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:22.957057 (+ 498us) server_negotiation.cc:407] Connection header received
0806 08:58:22.957281 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:22.957286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:22.957371 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:22.957471 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:22.958469 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.959243 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.960546 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:22.961049 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:22.961155 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:22.965226 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:46817: BlockingRecv error: recv got EOF from 127.22.98.1:46817 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:23.443466 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:23.434634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:46909)
0806 08:58:23.435192 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:23.435209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:23.435333 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:23.435723 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:23.435726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:23.436152 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:23.436313 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.436323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.437353 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.437357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.438324 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.438337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.439095 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.439100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.439236 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.439859 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:23.439879 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:23.443138 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":296,"threads_started":1}
W20260806 08:58:23.443516 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:23.434713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:46909 (local address 127.22.98.62:35903)
0806 08:58:23.434920 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:23.434924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:23.435298 (+ 374us) server_negotiation.cc:407] Connection header received
0806 08:58:23.435472 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:23.435477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:23.435580 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:23.435670 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:23.436460 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.437195 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.438478 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.438954 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.439061 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.443346 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:46909: BlockingRecv error: recv got EOF from 127.22.98.2:46909 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:23.915990 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:23.907015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:33773)
0806 08:58:23.907516 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:23.907564 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:23.907686 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:23.907968 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:23.907971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:23.908427 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:23.908720 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.908732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.909897 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.909901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.911117 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.911132 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.912144 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.912150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.912321 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.913011 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:23.913029 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:23.915732 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":279,"threads_started":1}
W20260806 08:58:23.916188 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:23.907072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:33773 (local address 127.22.98.62:35903)
0806 08:58:23.907319 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:23.907326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:23.907668 (+ 342us) server_negotiation.cc:407] Connection header received
0806 08:58:23.907783 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:23.907787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:23.907852 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:23.907925 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:23.908897 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.909689 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.911266 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.911983 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.912106 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.915946 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:33773: BlockingRecv error: recv got EOF from 127.22.98.3:33773 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:23 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:23.975072 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:23.966277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:39143)
0806 08:58:23.966871 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:23.966888 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:23.967007 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:23.967385 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:23.967389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:23.967965 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:23.968142 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.968152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.969204 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.969208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.970163 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:23.970175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.970895 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.970900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:23.971021 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.971697 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:23.971722 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:23.974743 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":324,"threads_started":1}
W20260806 08:58:23.975297 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:23.966384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:39143 (local address 127.22.98.62:35903)
0806 08:58:23.966552 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:23.966557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:23.966968 (+ 411us) server_negotiation.cc:407] Connection header received
0806 08:58:23.967152 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:23.967156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:23.967243 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:23.967333 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:23.968282 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.969047 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.970296 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:23.970715 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:23.970818 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:23.975004 (+ 4186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:39143: BlockingRecv error: recv got EOF from 127.22.98.1:39143 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:24.454550 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:24.444485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:49009)
0806 08:58:24.445094 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:24.445111 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:24.445228 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:24.445631 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:24.445662 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:24.446142 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:24.446320 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.446330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.447615 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.447619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.448935 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.448950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.449933 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.449937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.450056 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.450687 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:24.450707 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:24.454219 (+ 3512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":354,"threads_started":1}
W20260806 08:58:24.454694 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:24.444688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:49009 (local address 127.22.98.62:35903)
0806 08:58:24.444847 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:24.444851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:24.445212 (+ 361us) server_negotiation.cc:407] Connection header received
0806 08:58:24.445370 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:24.445374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:24.445441 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:24.445531 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:24.446469 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.447418 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.449100 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.449771 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.449878 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.454458 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:49009: BlockingRecv error: recv got EOF from 127.22.98.2:49009 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:24.925374 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:24.916997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:59245)
0806 08:58:24.917544 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:24.917566 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:24.917689 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:24.917985 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:24.917990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:24.918408 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:24.918574 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.918583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.919518 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.919522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.920485 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.920498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.921373 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.921379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.921496 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.922045 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:24.922064 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:24.925067 (+ 3003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":287,"threads_started":1}
W20260806 08:58:24.925477 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:24.917001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59245 (local address 127.22.98.62:35903)
0806 08:58:24.917196 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:24.917203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:24.917644 (+ 441us) server_negotiation.cc:407] Connection header received
0806 08:58:24.917797 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:24.917801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:24.917867 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:24.917944 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:24.918690 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.919395 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.920603 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.921127 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.921217 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.925292 (+ 4075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:59245: BlockingRecv error: recv got EOF from 127.22.98.3:59245 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:24 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:24.984243 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:24.976149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:54677)
0806 08:58:24.976629 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:24.976653 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:24.976774 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:24.977058 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:24.977061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:24.977444 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:24.977638 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.977649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.978650 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.978655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.979613 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:24.979625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.980333 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.980337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:24.980460 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.981042 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:24.981062 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:24.983940 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":253,"threads_started":1}
W20260806 08:58:24.984588 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:24.976218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54677 (local address 127.22.98.62:35903)
0806 08:58:24.976577 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:24.976583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:24.976722 (+ 139us) server_negotiation.cc:407] Connection header received
0806 08:58:24.976858 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:24.976862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:24.976937 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:24.977017 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:24.977765 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.978507 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.979753 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:24.980190 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:24.980291 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:24.984167 (+ 3876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:54677: BlockingRecv error: recv got EOF from 127.22.98.1:54677 (error 108)
Metrics: {"server-negotiator.queue_time_us":196}
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:25.464293 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:25.455637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:44089)
0806 08:58:25.456185 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:25.456201 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:25.456343 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:25.456670 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:25.456676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:25.457073 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:25.457277 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.457287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.458539 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.458543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.459448 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.459460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.460377 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.460383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.460560 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.461159 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:25.461178 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:25.463913 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":345,"threads_started":1}
W20260806 08:58:25.464370 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:25.455738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:44089 (local address 127.22.98.62:35903)
0806 08:58:25.455977 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:25.455983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:25.456291 (+ 308us) server_negotiation.cc:407] Connection header received
0806 08:58:25.456462 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:25.456468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:25.456533 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:25.456617 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:25.457442 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.458394 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.459595 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.460226 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.460348 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.464157 (+ 3809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:44089: BlockingRecv error: recv got EOF from 127.22.98.2:44089 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.3@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:25.934367 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:25.926315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.3:35583)
0806 08:58:25.926755 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:25.926779 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:25.926897 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:25.927183 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:25.927186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:25.927627 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:25.927818 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.927827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.928795 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.928800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.929742 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.929754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.930399 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.930403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.930525 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.931088 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:25.931106 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:25.934003 (+ 2897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":232,"threads_started":1}
W20260806 08:58:25.934460 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:25.926397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:35583 (local address 127.22.98.62:35903)
0806 08:58:25.926622 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:25.926629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:25.926853 (+ 224us) server_negotiation.cc:407] Connection header received
0806 08:58:25.927008 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:25.927011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:25.927076 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:25.927155 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:25.927926 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.928656 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.929869 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.930273 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.930367 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.934234 (+ 3867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.3:35583: BlockingRecv error: recv got EOF from 127.22.98.3:35583 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:25 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.1@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:25.994918 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:25.985419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.1:38231)
0806 08:58:25.985935 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:25.985952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:25.986063 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:25.986494 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:25.986498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:25.986998 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:25.987313 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.987326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.988565 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.988570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.989780 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:25.989794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.990469 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.990474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:25.990588 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.991254 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:25.991273 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:25.994651 (+ 3378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":286,"threads_started":1}
W20260806 08:58:25.995016 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:25.985499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:38231 (local address 127.22.98.62:35903)
0806 08:58:25.985667 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:25.985671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:25.986015 (+ 344us) server_negotiation.cc:407] Connection header received
0806 08:58:25.986201 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:25.986206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:25.986288 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:25.986379 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:25.987475 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.988364 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.989903 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:25.990341 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:25.990437 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:25.994815 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.1:38231: BlockingRecv error: recv got EOF from 127.22.98.1:38231 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 08:58:26 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 08:58:26 dist-test-slave-jpj1 krb5kdc[2188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.98.2@KRBTEST.COM for oryx/127.22.98.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 08:58:26.473803 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:26.465463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:35903 (local address 127.22.98.2:37231)
0806 08:58:26.466010 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:26.466026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:26.466157 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:26.466485 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:26.466490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:26.466944 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:26.467142 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:26.467152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.468223 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.468228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:26.469142 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:26.469155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.469929 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.469935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:26.470093 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:26.470674 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:26.470692 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:26.473445 (+ 2753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.98.62:35903: Server oryx/127.22.98.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":299,"threads_started":1}
W20260806 08:58:26.473861 3160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:26.465583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:37231 (local address 127.22.98.62:35903)
0806 08:58:26.465860 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:26.465866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:26.466103 (+ 237us) server_negotiation.cc:407] Connection header received
0806 08:58:26.466270 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:26.466275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:26.466346 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:26.466430 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:26.467286 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.468072 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.469283 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.469785 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.469900 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:26.473647 (+ 3747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.98.2:37231: BlockingRecv error: recv got EOF from 127.22.98.2:37231 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 08:58:26.768340 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2962
I20260806 08:58:26.781594 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:26.769601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35678 (local address 127.22.98.1:39667)
0806 08:58:26.770269 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:26.770276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:26.770303 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:58:26.770362 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:26.770369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:26.770590 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:26.770760 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:26.772322 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.773393 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.774609 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.775166 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.775297 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:26.778121 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:26.778130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:26.778145 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:26.778179 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:26.780853 (+ 2674us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 08:58:26.781177 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35678: Request ticket server kudu/127.22.98.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":278,"threads_started":1}
W20260806 08:58:26.781662 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2962: Not authorized: Client connection negotiation failed: client connection to 127.22.98.1:39667: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 08:58:26.781817 3566 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35678: Request ticket server kudu/127.22.98.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 08:58:26.781900 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2962
I20260806 08:58:26.811014 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3162
I20260806 08:58:26.823213 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:26.812192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.22.98.2:36789)
0806 08:58:26.812862 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:26.812870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:26.812903 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:58:26.812966 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:26.812973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:26.813203 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:26.813356 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:26.814579 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.815700 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.816947 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.817509 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.817643 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:26.820106 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:26.820115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:26.820130 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:26.820170 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:26.822678 (+ 2508us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 08:58:26.822943 (+ 265us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47222: Request ticket server kudu/127.22.98.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":315,"threads_started":1}
W20260806 08:58:26.823319 3567 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47222: Request ticket server kudu/127.22.98.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 08:58:26.823330 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3162: Not authorized: Client connection negotiation failed: client connection to 127.22.98.2:36789: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 08:58:26.823508 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 3162
I20260806 08:58:26.845520 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3336
I20260806 08:58:26.857520 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:26.846489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.22.98.3:37865)
0806 08:58:26.847049 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:26.847057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:26.847085 (+ 28us) server_negotiation.cc:407] Connection header received
0806 08:58:26.847149 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:26.847156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:26.847385 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:26.847579 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:26.848894 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.849941 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.851157 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:26.851820 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:26.851936 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:26.854437 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:26.854446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:26.854463 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:26.854503 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:26.857008 (+ 2505us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 08:58:26.857279 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54096: Request ticket server kudu/127.22.98.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":277,"threads_started":1}
W20260806 08:58:26.857625 3569 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54096: Request ticket server kudu/127.22.98.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 08:58:26.857826 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3336: Not authorized: Client connection negotiation failed: client connection to 127.22.98.3:37865: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.98.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 08:58:26.857972 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 3336
I20260806 08:58:26.883405 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 2869
I20260806 08:58:26.884986 2951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:27.002959 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 2869
2026-08-06T08:58:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22741 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T08:58:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:27Z Disabled control of system clock
I20260806 08:58:27.095201 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40253
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:40253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 08:58:27.447822 3577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:27.448158 3577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:27.448290 3577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:27.456123 3577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:27.456255 3577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:27.456305 3577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:27.456395 3577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:27.456458 3577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:27.466562 3577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:40253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:27.468075 3577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:27.469992 3577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:27.482481 3586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:27.482560 3583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:27.482540 3584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:27.483225 3577 server_base.cc:1034] running on GCE node
I20260806 08:58:27.484120 3577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:27.485613 3577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:27.486850 3577 hybrid_clock.cc:648] HybridClock initialized: now 1786006707486800 us; error 49 us; skew 500 ppm
I20260806 08:58:27.487274 3577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:27.492883 3577 webserver.cc:460] Webserver started at http://127.22.98.62:41599/ using document root <none> and password file <none>
I20260806 08:58:27.493600 3577 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:27.493692 3577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:27.494035 3577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:27.497095 3577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "95f83a52c57b4d8fb92a87c9a598f489"
format_stamp: "Formatted at 2026-08-06 08:58:27 on dist-test-slave-jpj1"
server_key: "29ffb9870aa520ce3f52cc5a853c30f0"
server_key_iv: "dba3884a10b51a1f579bd3e358ae5668"
server_key_version: "encryptionkey@0"
I20260806 08:58:27.497885 3577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "95f83a52c57b4d8fb92a87c9a598f489"
format_stamp: "Formatted at 2026-08-06 08:58:27 on dist-test-slave-jpj1"
server_key: "29ffb9870aa520ce3f52cc5a853c30f0"
server_key_iv: "dba3884a10b51a1f579bd3e358ae5668"
server_key_version: "encryptionkey@0"
I20260806 08:58:27.503218 3577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 08:58:27.506894 3592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:27.508414 3577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 08:58:27.508574 3577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "95f83a52c57b4d8fb92a87c9a598f489"
format_stamp: "Formatted at 2026-08-06 08:58:27 on dist-test-slave-jpj1"
server_key: "29ffb9870aa520ce3f52cc5a853c30f0"
server_key_iv: "dba3884a10b51a1f579bd3e358ae5668"
server_key_version: "encryptionkey@0"
I20260806 08:58:27.508782 3577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:27.524806 3577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:27.525695 3577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:27.525970 3577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:27.544533 3577 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:40253
I20260806 08:58:27.544552 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:40253 every 8 connection(s)
I20260806 08:58:27.545934 3577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:58:27.549842 3659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:27.551851 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 3577
I20260806 08:58:27.552081 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:58:27.552901 22920 external_mini_cluster.cc:1428] Setting key 03d593ad208f0ae41578e670af161ada
I20260806 08:58:27.562443 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489: Bootstrap starting.
I20260806 08:58:27.566046 3659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:27.567061 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:27.555107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.22.98.62:40253)
0806 08:58:27.556198 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:27.556210 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:27.556250 (+ 40us) server_negotiation.cc:407] Connection header received
0806 08:58:27.557872 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:27.557894 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:27.558336 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:27.558659 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:27.559324 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:27.560789 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:27.562585 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:27.563318 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:27.563452 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:27.564762 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:27.564770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:27.564784 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:27.565333 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:27.565430 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:27.565451 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:27.565536 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:27.565765 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 08:58:27.566077 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":422,"threads_started":1}
I20260806 08:58:27.567137 3659 log.cc:824] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:27.571448 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489: No bootstrap required, opened a new log
I20260806 08:58:27.578420 3659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } }
I20260806 08:58:27.578751 3659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:27.578824 3659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f83a52c57b4d8fb92a87c9a598f489, State: Initialized, Role: FOLLOWER
I20260806 08:58:27.579310 3659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } }
I20260806 08:58:27.579504 3659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:58:27.579638 3659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:58:27.579774 3659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:27.581969 3659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } }
I20260806 08:58:27.582424 3659 leader_election.cc:302] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f83a52c57b4d8fb92a87c9a598f489; no voters:
I20260806 08:58:27.582948 3659 leader_election.cc:288] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:58:27.583099 3666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:27.583791 3666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [term 1 LEADER]: Becoming Leader. State: Replica: 95f83a52c57b4d8fb92a87c9a598f489, State: Running, Role: LEADER
I20260806 08:58:27.584295 3666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } }
I20260806 08:58:27.584949 3659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:58:27.587023 3669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f83a52c57b4d8fb92a87c9a598f489. Latest consensus state: current_term: 1 leader_uuid: "95f83a52c57b4d8fb92a87c9a598f489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } } }
I20260806 08:58:27.587395 3669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:27.587054 3667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95f83a52c57b4d8fb92a87c9a598f489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f83a52c57b4d8fb92a87c9a598f489" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40253 } } }
I20260806 08:58:27.588641 3667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489 [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:27.592521 3674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:58:27.597728 3674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:58:27.606178 3674 catalog_manager.cc:1347] Generated new cluster ID: 39910be0bbc5465d97bee92a6d1c4d05
I20260806 08:58:27.606292 3674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:58:27.628633 3674 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:58:27.629987 3674 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:58:27.638041 3674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f83a52c57b4d8fb92a87c9a598f489: Generated new TSK 0
I20260806 08:58:27.638959 3674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:58:27.653227 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--builtin_ntp_servers=127.22.98.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 08:58:28.012550 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:28.013041 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:28.013188 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:28.023805 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:28.023947 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:28.024124 3693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:58:28.036767 3693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:28.038354 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:28.040712 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:28.055868 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.056102 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.055903 3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:28.056802 3693 server_base.cc:1034] running on GCE node
I20260806 08:58:28.057456 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:28.058437 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:28.059652 3693 hybrid_clock.cc:648] HybridClock initialized: now 1786006708059589 us; error 53 us; skew 500 ppm
I20260806 08:58:28.059984 3693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:28.062318 3693 webserver.cc:460] Webserver started at http://127.22.98.1:34619/ using document root <none> and password file <none>
I20260806 08:58:28.063056 3693 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:28.063165 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:28.063481 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:28.066529 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "eb17a7d6d151458bbfb8b5e776dc0448"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e09529a15c5d9d2e204ae0922da50296"
server_key_iv: "911d11ef49808810ccf5f77f05663a8c"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.067350 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "eb17a7d6d151458bbfb8b5e776dc0448"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e09529a15c5d9d2e204ae0922da50296"
server_key_iv: "911d11ef49808810ccf5f77f05663a8c"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.072911 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:58:28.076604 3713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.078094 3693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:58:28.078263 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "eb17a7d6d151458bbfb8b5e776dc0448"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e09529a15c5d9d2e204ae0922da50296"
server_key_iv: "911d11ef49808810ccf5f77f05663a8c"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.078464 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:28.101630 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:28.102557 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:28.102851 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:28.103922 3693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:28.105453 3693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:28.105541 3693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.105650 3693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:28.105726 3693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.138379 3693 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:35733
I20260806 08:58:28.138439 3863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:35733 every 8 connection(s)
I20260806 08:58:28.139755 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:58:28.147292 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 3693
I20260806 08:58:28.147508 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:58:28.148244 22920 external_mini_cluster.cc:1428] Setting key cabf038b7677b7040a60cab8078f28bc
I20260806 08:58:28.151453 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--builtin_ntp_servers=127.22.98.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 08:58:28.153482 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:28.142425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:52235 (local address 127.22.98.62:40253)
0806 08:58:28.142899 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:28.142903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:28.144186 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 08:58:28.146033 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:28.146039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:28.146132 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:28.146241 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:28.147408 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.148274 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.150312 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.150899 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.151043 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:28.152553 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:28.152560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:28.152564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:28.152587 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:28.152639 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:28.152654 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:28.152747 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:28.153019 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 08:58:28.153123 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":238,"threads_started":1}
I20260806 08:58:28.154515 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:28.143007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40253 (local address 127.22.98.1:52235)
0806 08:58:28.144024 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:28.144069 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:28.145756 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:28.146483 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:28.146492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:28.146593 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:28.147230 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:28.147247 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.148442 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.148446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:28.150141 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:28.150159 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.151135 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.151140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:28.151366 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:28.152175 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:28.152177 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:28.152224 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:28.152324 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:28.152804 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:28.152814 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:28.152821 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 08:58:28.152993 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 08:58:28.153354 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":289,"threads_started":1}
I20260806 08:58:28.156189 3865 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40253
I20260806 08:58:28.156529 3865 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:28.157344 3865 heartbeater.cc:511] Master 127.22.98.62:40253 requested a full tablet report, sending...
I20260806 08:58:28.159859 3611 ts_manager.cc:194] Registered new tserver with Master: eb17a7d6d151458bbfb8b5e776dc0448 (127.22.98.1:35733)
I20260806 08:58:28.161580 3611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.1:52235
W20260806 08:58:28.495385 3870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:28.495723 3870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:28.495824 3870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:28.502990 3870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:28.503096 3870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:28.503201 3870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:58:28.512470 3870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:28.513784 3870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:28.515683 3870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:28.529496 3875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.529455 3878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.529520 3876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:28.530406 3870 server_base.cc:1034] running on GCE node
I20260806 08:58:28.531071 3870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:28.532114 3870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:28.533336 3870 hybrid_clock.cc:648] HybridClock initialized: now 1786006708533278 us; error 53 us; skew 500 ppm
I20260806 08:58:28.533682 3870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:28.536157 3870 webserver.cc:460] Webserver started at http://127.22.98.2:38229/ using document root <none> and password file <none>
I20260806 08:58:28.536890 3870 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:28.537007 3870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:28.537339 3870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:28.540400 3870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "1b48e60146f74c60bd955b615e2275de"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e98671e7ae022e8a9018e9d404d5d956"
server_key_iv: "4cd5b326865cf20b13188624ec6e3e4f"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.541210 3870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "1b48e60146f74c60bd955b615e2275de"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e98671e7ae022e8a9018e9d404d5d956"
server_key_iv: "4cd5b326865cf20b13188624ec6e3e4f"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.546689 3870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:58:28.550382 3891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.551903 3870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 08:58:28.552064 3870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "1b48e60146f74c60bd955b615e2275de"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "e98671e7ae022e8a9018e9d404d5d956"
server_key_iv: "4cd5b326865cf20b13188624ec6e3e4f"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.552273 3870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:28.571513 3870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:28.572450 3870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:28.572722 3870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:28.573738 3870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:28.575235 3870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:28.575344 3870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.575452 3870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:28.575523 3870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:28.607615 3870 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:36745
I20260806 08:58:28.607681 4039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:36745 every 8 connection(s)
I20260806 08:58:28.608958 3870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:58:28.610720 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 3870
I20260806 08:58:28.610911 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:58:28.611406 22920 external_mini_cluster.cc:1428] Setting key c3ac5bcd842804a0ba32c3fe2efff37c
I20260806 08:58:28.614285 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--builtin_ntp_servers=127.22.98.20:37187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 08:58:28.621853 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:28.611880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:45767 (local address 127.22.98.62:40253)
0806 08:58:28.612098 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:28.612105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:28.613752 (+ 1647us) server_negotiation.cc:407] Connection header received
0806 08:58:28.615524 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:28.615530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:28.615634 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:28.615738 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:28.616749 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.617668 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.619154 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.619712 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.619831 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:28.621098 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:28.621104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:28.621108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:28.621130 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:28.621171 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:28.621185 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:28.621283 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:28.621447 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 08:58:28.621540 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 08:58:28.622802 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:28.612430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40253 (local address 127.22.98.2:45767)
0806 08:58:28.613595 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:28.613650 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:28.615256 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:28.615928 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:28.615936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:28.616021 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:28.616589 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:28.616605 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.617791 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.617794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:28.619016 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:28.619029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:28.619919 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:28.619925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:28.620129 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:28.620785 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:28.620787 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:28.620828 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:28.620913 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:28.621277 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:28.621283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:28.621288 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:58:28.621419 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 08:58:28.621717 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":363,"threads_started":1}
I20260806 08:58:28.624225 4040 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40253
I20260806 08:58:28.624514 4040 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:28.625280 4040 heartbeater.cc:511] Master 127.22.98.62:40253 requested a full tablet report, sending...
I20260806 08:58:28.626660 3611 ts_manager.cc:194] Registered new tserver with Master: 1b48e60146f74c60bd955b615e2275de (127.22.98.2:36745)
I20260806 08:58:28.627462 3611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.2:45767
W20260806 08:58:28.952280 4045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:28.952576 4045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:28.952705 4045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:28.960340 4045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:28.960454 4045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:28.960598 4045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:58:28.969548 4045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:37187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:28.970851 4045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:28.972643 4045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:28.985733 4050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.985800 4051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:28.986068 4056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:28.987303 4045 server_base.cc:1034] running on GCE node
I20260806 08:58:28.987970 4045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:28.988891 4045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:28.990082 4045 hybrid_clock.cc:648] HybridClock initialized: now 1786006708990009 us; error 65 us; skew 500 ppm
I20260806 08:58:28.990371 4045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:28.992642 4045 webserver.cc:460] Webserver started at http://127.22.98.3:36887/ using document root <none> and password file <none>
I20260806 08:58:28.993340 4045 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:28.993427 4045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:28.993702 4045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:28.996604 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "8fa3c58182f9484fba42ccb4820510c5"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "4dca5d619458e98e266fd33af7d2df18"
server_key_iv: "aee8fc67ec1f54ea70cf0f7f09b798ae"
server_key_version: "encryptionkey@0"
I20260806 08:58:28.997364 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "8fa3c58182f9484fba42ccb4820510c5"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "4dca5d619458e98e266fd33af7d2df18"
server_key_iv: "aee8fc67ec1f54ea70cf0f7f09b798ae"
server_key_version: "encryptionkey@0"
I20260806 08:58:29.002647 4045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:58:29.006255 4064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:29.007607 4045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 08:58:29.007755 4045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "8fa3c58182f9484fba42ccb4820510c5"
format_stamp: "Formatted at 2026-08-06 08:58:28 on dist-test-slave-jpj1"
server_key: "4dca5d619458e98e266fd33af7d2df18"
server_key_iv: "aee8fc67ec1f54ea70cf0f7f09b798ae"
server_key_version: "encryptionkey@0"
I20260806 08:58:29.007948 4045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:29.032236 4045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:29.033097 4045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:29.033344 4045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:29.034317 4045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:29.035938 4045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:29.036029 4045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:29.036116 4045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:29.036171 4045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:29.067332 4045 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:34603
I20260806 08:58:29.067428 4219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:34603 every 8 connection(s)
I20260806 08:58:29.068665 4045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:58:29.073351 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4045
I20260806 08:58:29.073524 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:58:29.073971 22920 external_mini_cluster.cc:1428] Setting key 67e0774bbe72c3a40c45f910ddf8f532
I20260806 08:58:29.081607 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:29.071285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:41519 (local address 127.22.98.62:40253)
0806 08:58:29.071469 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:29.071474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:29.072956 (+ 1482us) server_negotiation.cc:407] Connection header received
0806 08:58:29.074994 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:29.075000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:29.075093 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:29.075180 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:29.076516 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.077291 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.078863 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.079264 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.079376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:29.080768 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:29.080773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:29.080777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:29.080801 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:29.080834 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:29.080846 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:29.080920 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:29.081167 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 08:58:29.081276 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 08:58:29.082729 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:29.071823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40253 (local address 127.22.98.3:41519)
0806 08:58:29.072776 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:29.072822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:29.074702 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:29.075483 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:29.075496 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:29.075657 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:29.076347 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:29.076367 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.077452 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.077456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:29.078715 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:29.078729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.079527 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.079565 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:29.079795 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:29.080436 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:29.080439 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:29.080481 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:29.080572 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:29.080936 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:29.080948 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:29.080956 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 08:58:29.081124 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 08:58:29.081563 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260806 08:58:29.084249 4220 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40253
I20260806 08:58:29.084527 4220 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:29.085325 4220 heartbeater.cc:511] Master 127.22.98.62:40253 requested a full tablet report, sending...
I20260806 08:58:29.086774 3611 ts_manager.cc:194] Registered new tserver with Master: 8fa3c58182f9484fba42ccb4820510c5 (127.22.98.3:34603)
I20260806 08:58:29.087724 3611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.3:41519
I20260806 08:58:29.089804 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 08:58:29.096203 3868 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:29.094672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.22.98.62:40253)
0806 08:58:29.094885 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:29.094889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:29.095277 (+ 388us) server_negotiation.cc:407] Connection header received
0806 08:58:29.095427 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:29.095431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:29.095511 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:29.095626 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:29.095966 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48248: BlockingRecv error: recv got EOF from 127.0.0.1:48248 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 08:58:29.097082 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3693
I20260806 08:58:29.104779 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:29.097983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.22.98.1:35733)
0806 08:58:29.098490 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:29.098498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:29.098525 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:58:29.098581 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:29.098587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:29.098834 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:29.098960 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:29.099336 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.100435 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.101699 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.102269 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.102375 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:29.103308 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:29.103317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:29.103332 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:29.104007 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:29.104055 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:29.104077 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:29.104165 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:29.104326 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 08:58:29.104439 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":231,"threads_started":1}
I20260806 08:58:29.106041 3858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:29.186276 3865 heartbeater.cc:503] Master 127.22.98.62:40253 was elected leader, sending a full tablet report...
I20260806 08:58:29.187600 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 3693
I20260806 08:58:29.207824 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3870
W20260806 08:58:29.207474 3594 connection.cc:535] server connection from 127.22.98.1:52235 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 08:58:29.216099 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:29.208795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.22.98.2:36745)
0806 08:58:29.209348 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:29.209356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:29.209384 (+ 28us) server_negotiation.cc:407] Connection header received
0806 08:58:29.209448 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:29.209454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:29.209691 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:29.209851 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:29.210265 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.211270 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.212739 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.213364 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.213486 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:29.214548 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:29.214556 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:29.214572 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:29.215200 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:29.215253 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:29.215274 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:29.215382 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:29.215583 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 08:58:29.215749 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
I20260806 08:58:29.217440 4035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:29.299726 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 3870
I20260806 08:58:29.323956 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4045
I20260806 08:58:29.332074 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:29.325013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.22.98.3:34603)
0806 08:58:29.325660 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:29.325667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:29.325699 (+ 32us) server_negotiation.cc:407] Connection header received
0806 08:58:29.325792 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:29.325799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:29.326020 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:29.326146 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:29.326564 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.327656 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.328847 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:29.329420 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:29.329517 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:29.330494 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:29.330503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:29.330518 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:29.331121 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:29.331167 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:29.331188 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:29.331427 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:29.331525 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 08:58:29.331684 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":328,"threads_started":1}
I20260806 08:58:29.333429 4215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:29.416389 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4045
I20260806 08:58:29.440095 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 3577
I20260806 08:58:29.441385 3654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:29.544533 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 3577
2026-08-06T08:58:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2536 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T08:58:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:29Z Disabled control of system clock
I20260806 08:58:29.628930 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:44853
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:40969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:44853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 08:58:29.968458 4247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:29.968755 4247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:29.968839 4247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:29.976082 4247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:29.976204 4247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:29.976244 4247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:29.976313 4247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:29.976358 4247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:29.985742 4247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:44853
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:44853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:29.987093 4247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:29.988962 4247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:30.001163 4257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:30.001089 4255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:30.001149 4259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:30.001989 4247 server_base.cc:1034] running on GCE node
I20260806 08:58:30.002864 4247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:30.004494 4247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:30.005762 4247 hybrid_clock.cc:648] HybridClock initialized: now 1786006710005694 us; error 61 us; skew 500 ppm
I20260806 08:58:30.006167 4247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:30.008904 4247 webserver.cc:460] Webserver started at http://127.22.98.62:42421/ using document root <none> and password file <none>
I20260806 08:58:30.009692 4247 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:30.009811 4247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:30.010150 4247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:30.013351 4247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "d6956b3d813bd61531b4ec6d9c7b223c"
server_key_iv: "59ba2e3e010333c3179b2de56b872fab"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.014214 4247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "d6956b3d813bd61531b4ec6d9c7b223c"
server_key_iv: "59ba2e3e010333c3179b2de56b872fab"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.020090 4247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:58:30.023932 4267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:30.025444 4247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:58:30.025616 4247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "d6956b3d813bd61531b4ec6d9c7b223c"
server_key_iv: "59ba2e3e010333c3179b2de56b872fab"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.025832 4247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:30.031705 4247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:30.032505 4247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:30.032780 4247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:30.050450 4247 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:44853
I20260806 08:58:30.050469 4334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:44853 every 8 connection(s)
I20260806 08:58:30.051884 4247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:58:30.054988 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4247
I20260806 08:58:30.055162 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:58:30.055946 22920 external_mini_cluster.cc:1428] Setting key fcbf4117ab11fc3f1b9ec647b6510816
I20260806 08:58:30.055974 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:30.065153 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:30.057990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.22.98.62:44853)
0806 08:58:30.059124 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:30.059139 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:30.059192 (+ 53us) server_negotiation.cc:407] Connection header received
0806 08:58:30.060931 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:30.060961 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:30.061542 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:30.061999 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:30.063057 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:30.063065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:30.063087 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:30.063107 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:30.063186 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:30.063212 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:30.063461 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:30.063652 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 08:58:30.063939 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":358,"threads_started":1}
I20260806 08:58:30.070847 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e: Bootstrap starting.
I20260806 08:58:30.073954 4336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:30.075137 4336 log.cc:824] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:30.078014 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e: No bootstrap required, opened a new log
I20260806 08:58:30.085600 4336 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } }
I20260806 08:58:30.086030 4336 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:30.086141 4336 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf2e8ea6a88487c889086e2d3ca2a5e, State: Initialized, Role: FOLLOWER
I20260806 08:58:30.086735 4336 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } }
I20260806 08:58:30.086966 4336 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:58:30.087097 4336 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:58:30.087253 4336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:30.089483 4336 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } }
I20260806 08:58:30.090015 4336 leader_election.cc:302] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdf2e8ea6a88487c889086e2d3ca2a5e; no voters:
I20260806 08:58:30.090610 4336 leader_election.cc:288] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:58:30.090768 4342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:30.091501 4342 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [term 1 LEADER]: Becoming Leader. State: Replica: bdf2e8ea6a88487c889086e2d3ca2a5e, State: Running, Role: LEADER
I20260806 08:58:30.092161 4342 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } }
I20260806 08:58:30.093026 4336 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:58:30.094858 4344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdf2e8ea6a88487c889086e2d3ca2a5e. Latest consensus state: current_term: 1 leader_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } } }
I20260806 08:58:30.095242 4344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:30.096050 4343 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf2e8ea6a88487c889086e2d3ca2a5e" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 44853 } } }
I20260806 08:58:30.096297 4352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:58:30.097355 4343 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:30.102907 4352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:58:30.112270 4352 catalog_manager.cc:1347] Generated new cluster ID: 5396a3647e8643a7b255afffcef63366
I20260806 08:58:30.112394 4352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:58:30.139024 4352 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:58:30.139573 4352 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:58:30.146327 4352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdf2e8ea6a88487c889086e2d3ca2a5e: Generated new TSK 0
I20260806 08:58:30.147164 4352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 08:58:30.156083 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--builtin_ntp_servers=127.22.98.20:40969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 08:58:30.520323 4367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:30.520615 4367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:30.520707 4367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:30.527902 4367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:30.528021 4367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:30.528172 4367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:58:30.537556 4367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:30.538887 4367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:30.540699 4367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:30.553469 4373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:30.553514 4374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:30.553536 4376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:30.554311 4367 server_base.cc:1034] running on GCE node
I20260806 08:58:30.554945 4367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:30.555979 4367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:30.557260 4367 hybrid_clock.cc:648] HybridClock initialized: now 1786006710557189 us; error 57 us; skew 500 ppm
I20260806 08:58:30.557595 4367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:30.560057 4367 webserver.cc:460] Webserver started at http://127.22.98.1:38017/ using document root <none> and password file <none>
I20260806 08:58:30.560777 4367 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:30.560895 4367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:30.561218 4367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:30.564132 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "bc4dfd708a974609bea23e9284b78f94"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "3419b0dcc1fb5d161baab284a2c255e2"
server_key_iv: "0b8f76855b3ebef7257c9525cdb834a3"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.564944 4367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "bc4dfd708a974609bea23e9284b78f94"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "3419b0dcc1fb5d161baab284a2c255e2"
server_key_iv: "0b8f76855b3ebef7257c9525cdb834a3"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.570294 4367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:58:30.573863 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:30.575307 4367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 08:58:30.575475 4367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "bc4dfd708a974609bea23e9284b78f94"
format_stamp: "Formatted at 2026-08-06 08:58:30 on dist-test-slave-jpj1"
server_key: "3419b0dcc1fb5d161baab284a2c255e2"
server_key_iv: "0b8f76855b3ebef7257c9525cdb834a3"
server_key_version: "encryptionkey@0"
I20260806 08:58:30.575760 4367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:30.581540 4367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:30.582376 4367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:30.582638 4367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:30.583647 4367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:30.585139 4367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:30.585237 4367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:30.585358 4367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:30.585429 4367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:30.616022 4367 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:40851
I20260806 08:58:30.616081 4543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:40851 every 8 connection(s)
I20260806 08:58:30.617364 4367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:58:30.619184 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4367
I20260806 08:58:30.619386 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:58:30.620281 22920 external_mini_cluster.cc:1428] Setting key 1e339af6ebd1773c318098ae88e87fc8
I20260806 08:58:30.623294 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--builtin_ntp_servers=127.22.98.20:40969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 08:58:30.626062 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:30.620158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:53191 (local address 127.22.98.62:44853)
0806 08:58:30.620585 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:30.620590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:30.622076 (+ 1486us) server_negotiation.cc:407] Connection header received
0806 08:58:30.623965 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:30.623970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:30.624045 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:30.624156 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:30.625371 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:30.625377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:30.625381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:30.625400 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:30.625453 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:30.625467 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:30.625595 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:30.625715 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 08:58:30.625777 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":224,"threads_started":1}
I20260806 08:58:30.626924 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:30.620845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44853 (local address 127.22.98.1:53191)
0806 08:58:30.621900 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:30.621958 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:30.623694 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:30.624334 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:30.624342 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:30.624434 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:30.625071 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:30.625073 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:30.625112 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:30.625201 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:30.625551 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:30.625557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:30.625562 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 08:58:30.625794 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 08:58:30.625990 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":269,"threads_started":1}
I20260806 08:58:30.628463 4544 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44853
I20260806 08:58:30.628777 4544 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:30.629495 4544 heartbeater.cc:511] Master 127.22.98.62:44853 requested a full tablet report, sending...
I20260806 08:58:30.631712 4287 ts_manager.cc:194] Registered new tserver with Master: bc4dfd708a974609bea23e9284b78f94 (127.22.98.1:40851)
W20260806 08:58:30.966457 4550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:30.966756 4550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:30.966861 4550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:30.974140 4550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:30.974241 4550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:30.974356 4550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:58:30.983456 4550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:30.984790 4550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:30.986579 4550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:31.000012 4556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:30.999963 4558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:31.000088 4560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:31.000862 4550 server_base.cc:1034] running on GCE node
I20260806 08:58:31.001536 4550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:31.002506 4550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:31.003705 4550 hybrid_clock.cc:648] HybridClock initialized: now 1786006711003644 us; error 48 us; skew 500 ppm
I20260806 08:58:31.004017 4550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:31.006229 4550 webserver.cc:460] Webserver started at http://127.22.98.2:35905/ using document root <none> and password file <none>
I20260806 08:58:31.006906 4550 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:31.006996 4550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:31.007278 4550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:31.010195 4550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "95214451e8a04e1a94fb71ad53a2bcd9"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "70ddcf4157a6c312f1ed90f9a3d5ac3a"
server_key_iv: "116065b7f395b3dac2a9137c186d44ec"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.010963 4550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "95214451e8a04e1a94fb71ad53a2bcd9"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "70ddcf4157a6c312f1ed90f9a3d5ac3a"
server_key_iv: "116065b7f395b3dac2a9137c186d44ec"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.016229 4550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 08:58:31.019719 4566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.021114 4550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:58:31.021296 4550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "95214451e8a04e1a94fb71ad53a2bcd9"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "70ddcf4157a6c312f1ed90f9a3d5ac3a"
server_key_iv: "116065b7f395b3dac2a9137c186d44ec"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.021551 4550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:31.027302 4550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:31.028141 4550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:31.028403 4550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:31.029340 4550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:31.030821 4550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:31.030921 4550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.031025 4550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:31.031097 4550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.061097 4550 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:34395
I20260806 08:58:31.061129 4717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:34395 every 8 connection(s)
I20260806 08:58:31.062672 4550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:58:31.071720 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.065511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:42277 (local address 127.22.98.62:44853)
0806 08:58:31.065708 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.065714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.067461 (+ 1747us) server_negotiation.cc:407] Connection header received
0806 08:58:31.069278 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.069285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.069357 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.069450 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.070846 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:31.070851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:31.070855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:31.070874 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:31.070910 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:31.070922 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:31.071031 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:31.071256 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 08:58:31.071332 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 08:58:31.072685 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4550
I20260806 08:58:31.072872 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:58:31.072865 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.066286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44853 (local address 127.22.98.2:42277)
0806 08:58:31.067285 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:31.067326 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:31.068975 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:31.069737 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:31.069746 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:31.069847 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:31.070506 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:31.070508 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:31.070550 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:31.070650 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:31.071015 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:31.071026 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:31.071034 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 08:58:31.071332 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 08:58:31.071648 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":302,"threads_started":1}
I20260806 08:58:31.073464 22920 external_mini_cluster.cc:1428] Setting key 5af7e56b7d8ce938dbc7bad389ff8610
I20260806 08:58:31.074504 4718 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44853
I20260806 08:58:31.074880 4718 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:31.075639 4718 heartbeater.cc:511] Master 127.22.98.62:44853 requested a full tablet report, sending...
I20260806 08:58:31.076731 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--builtin_ntp_servers=127.22.98.20:40969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 08:58:31.077322 4287 ts_manager.cc:194] Registered new tserver with Master: 95214451e8a04e1a94fb71ad53a2bcd9 (127.22.98.2:34395)
W20260806 08:58:31.422425 4723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:31.422719 4723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:31.422866 4723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:31.430229 4723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:31.430349 4723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:31.430501 4723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:58:31.439808 4723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:44853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:31.441160 4723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:31.442981 4723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:31.456637 4733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:31.456660 4735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:31.456749 4731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:31.457432 4723 server_base.cc:1034] running on GCE node
I20260806 08:58:31.458032 4723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:31.458984 4723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:31.460192 4723 hybrid_clock.cc:648] HybridClock initialized: now 1786006711460123 us; error 61 us; skew 500 ppm
I20260806 08:58:31.460536 4723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 08:58:31.462708 4723 webserver.cc:460] Webserver started at http://127.22.98.3:41269/ using document root <none> and password file <none>
I20260806 08:58:31.463389 4723 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:31.463477 4723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:31.463809 4723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:31.466764 4723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "30db426ffde5441eb4b57ada3748521f"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "34e3a02ada9f0b4958e13d8924b3bcb9"
server_key_iv: "e548db0a619c3e1d410c2a0379aaaf56"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.467746 4723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "30db426ffde5441eb4b57ada3748521f"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "34e3a02ada9f0b4958e13d8924b3bcb9"
server_key_iv: "e548db0a619c3e1d410c2a0379aaaf56"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.473089 4723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 08:58:31.476573 4742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.477929 4723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 08:58:31.478089 4723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "30db426ffde5441eb4b57ada3748521f"
format_stamp: "Formatted at 2026-08-06 08:58:31 on dist-test-slave-jpj1"
server_key: "34e3a02ada9f0b4958e13d8924b3bcb9"
server_key_iv: "e548db0a619c3e1d410c2a0379aaaf56"
server_key_version: "encryptionkey@0"
I20260806 08:58:31.478300 4723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:31.484072 4723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:31.484928 4723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:31.485180 4723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:31.486111 4723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:31.487605 4723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:31.487705 4723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.487812 4723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:31.487890 4723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:31.518103 4723 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:40465
I20260806 08:58:31.518152 4888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:40465 every 8 connection(s)
I20260806 08:58:31.519522 4723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:58:31.525802 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4723
I20260806 08:58:31.525983 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:58:31.526564 22920 external_mini_cluster.cc:1428] Setting key 1ec98a00f0b5216372cb17a30e999693
I20260806 08:58:31.527985 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.522096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:49111 (local address 127.22.98.62:44853)
0806 08:58:31.522319 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.522324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.523685 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 08:58:31.525488 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.525499 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.525578 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.525675 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.527185 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:31.527196 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:31.527199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:31.527217 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:31.527251 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:31.527263 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:31.527380 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:31.527608 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 08:58:31.527678 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 08:58:31.529053 4895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.522585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:44853 (local address 127.22.98.3:49111)
0806 08:58:31.523494 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:31.523570 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:31.525200 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:31.525969 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:31.525981 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:31.526107 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:31.526848 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 08:58:31.526851 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:31.526892 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 08:58:31.526984 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:31.527364 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:31.527375 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:31.527383 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 08:58:31.527679 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 08:58:31.527955 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":271,"threads_started":1}
I20260806 08:58:31.530375 4890 heartbeater.cc:347] Connected to a master server at 127.22.98.62:44853
I20260806 08:58:31.530659 4890 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:31.531332 4890 heartbeater.cc:511] Master 127.22.98.62:44853 requested a full tablet report, sending...
I20260806 08:58:31.532728 4287 ts_manager.cc:194] Registered new tserver with Master: 30db426ffde5441eb4b57ada3748521f (127.22.98.3:40465)
I20260806 08:58:31.542351 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 08:58:31.548838 4548 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 08:58:31.547134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.22.98.62:44853)
0806 08:58:31.547350 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.547355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.547861 (+ 506us) server_negotiation.cc:407] Connection header received
0806 08:58:31.548049 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.548054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.548115 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.548205 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.548512 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34892: BlockingRecv error: recv got EOF from 127.0.0.1:34892 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 08:58:31.550006 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4367
I20260806 08:58:31.553678 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.551062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.22.98.1:40851)
0806 08:58:31.551612 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.551620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.551649 (+ 29us) server_negotiation.cc:407] Connection header received
0806 08:58:31.551723 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.551732 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.551983 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.552126 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.553009 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:31.553016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:31.553028 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:31.553048 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:31.553113 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:31.553130 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:31.553239 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:31.553292 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 08:58:31.553376 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":231,"threads_started":1}
I20260806 08:58:31.554893 4537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:31.635226 4544 heartbeater.cc:503] Master 127.22.98.62:44853 was elected leader, sending a full tablet report...
I20260806 08:58:31.636739 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4367
W20260806 08:58:31.661017 4272 connection.cc:535] server connection from 127.22.98.1:53191 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 08:58:31.661365 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4550
I20260806 08:58:31.664932 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.662321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.22.98.2:34395)
0806 08:58:31.662893 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.662902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.662932 (+ 30us) server_negotiation.cc:407] Connection header received
0806 08:58:31.663006 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.663013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.663232 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.663356 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.664271 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:31.664277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:31.664289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:31.664307 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:31.664347 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:31.664361 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:31.664464 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:31.664542 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 08:58:31.664623 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":258,"threads_started":1}
I20260806 08:58:31.666232 4710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:31.746734 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4550
I20260806 08:58:31.770753 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4723
I20260806 08:58:31.774501 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:31.771687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.22.98.3:40465)
0806 08:58:31.772248 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:31.772255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:31.772288 (+ 33us) server_negotiation.cc:407] Connection header received
0806 08:58:31.772352 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:31.772358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:31.772572 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:31.772715 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:31.773795 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:31.773802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:31.773813 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 08:58:31.773826 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:31.773868 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 08:58:31.773884 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:31.774038 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:31.774111 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 08:58:31.774220 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":247,"threads_started":1}
I20260806 08:58:31.775975 4882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:31.857913 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4723
I20260806 08:58:31.880795 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4247
I20260806 08:58:31.882151 4329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:31.984588 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4247
2026-08-06T08:58:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2441 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:58:32 dist-test-slave-jpj1 krb5kdc[4917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:58:32 dist-test-slave-jpj1 krb5kdc[4917](info): set up 2 sockets
Aug 06 08:58:32 dist-test-slave-jpj1 krb5kdc[4917](info): commencing operation
krb5kdc: starting...
W20260806 08:58:33.182756 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:58:33 dist-test-slave-jpj1 krb5kdc[4917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:58:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:58:33.487195 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40209
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:33.830210 4938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:33.830528 4938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:33.830637 4938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:33.838323 4938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:58:33.838434 4938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:33.838491 4938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:33.838548 4938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:58:33.838591 4938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:58:33.848798 4938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:40209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:33.850167 4938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:33.851477 4938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:33.863371 4945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:33.863313 4948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:33.863332 4946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:33.864208 4938 server_base.cc:1034] running on GCE node
I20260806 08:58:33.865090 4938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:33.866674 4938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:33.867931 4938 hybrid_clock.cc:648] HybridClock initialized: now 1786006713867872 us; error 45 us; skew 500 ppm
I20260806 08:58:33.868382 4938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:33 dist-test-slave-jpj1 krb5kdc[4917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:33.871728 4938 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:58:33.873296 4938 webserver.cc:460] Webserver started at http://127.22.98.62:46759/ using document root <none> and password file <none>
I20260806 08:58:33.873973 4938 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:33.874085 4938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:33.874403 4938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:33.877422 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "e71af442d493417fa3ff3bd6d9c5650f"
format_stamp: "Formatted at 2026-08-06 08:58:33 on dist-test-slave-jpj1"
server_key: "115d026137ff970f3805e76b0db9cc02"
server_key_iv: "0e3b322dce88cda55577cffe284c33eb"
server_key_version: "encryptionkey@0"
I20260806 08:58:33.878165 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "e71af442d493417fa3ff3bd6d9c5650f"
format_stamp: "Formatted at 2026-08-06 08:58:33 on dist-test-slave-jpj1"
server_key: "115d026137ff970f3805e76b0db9cc02"
server_key_iv: "0e3b322dce88cda55577cffe284c33eb"
server_key_version: "encryptionkey@0"
I20260806 08:58:33.883458 4938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 08:58:33.887046 4956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:33.888484 4938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 08:58:33.888653 4938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "e71af442d493417fa3ff3bd6d9c5650f"
format_stamp: "Formatted at 2026-08-06 08:58:33 on dist-test-slave-jpj1"
server_key: "115d026137ff970f3805e76b0db9cc02"
server_key_iv: "0e3b322dce88cda55577cffe284c33eb"
server_key_version: "encryptionkey@0"
I20260806 08:58:33.888844 4938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:33.903054 4938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:33.907128 4938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:33.907395 4938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:33.925416 4938 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:40209
I20260806 08:58:33.925462 5019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:40209 every 8 connection(s)
I20260806 08:58:33.926772 4938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:58:33.930579 5020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:33.935576 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 4938
I20260806 08:58:33.935808 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:58:33.936620 22920 external_mini_cluster.cc:1428] Setting key 3b77284b1dd5bd25122fcd412793e628
I20260806 08:58:33.942152 5020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: Bootstrap starting.
I20260806 08:58:33.945894 5020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:33.947182 5020 log.cc:824] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:33.950802 5020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: No bootstrap required, opened a new log
Aug 06 08:58:33 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:33.958722 5020 raft_consensus.cc:348] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } }
I20260806 08:58:33.959136 5020 raft_consensus.cc:374] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:33.959249 5020 raft_consensus.cc:729] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71af442d493417fa3ff3bd6d9c5650f, State: Initialized, Role: FOLLOWER
I20260806 08:58:33.959923 5020 consensus_queue.cc:260] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } }
I20260806 08:58:33.960178 5020 raft_consensus.cc:388] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:58:33.960321 5020 raft_consensus.cc:482] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:58:33.960491 5020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:33.960770 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:33.938826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.22.98.62:40209)
0806 08:58:33.939723 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:33.939735 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:33.939781 (+ 46us) server_negotiation.cc:407] Connection header received
0806 08:58:33.941480 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:33.941503 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:33.941901 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:33.942223 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:33.944212 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:33.945574 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:33.947394 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:33.948472 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:33.948676 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:33.953214 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:33.953223 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:33.953239 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:33.953278 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:33.956460 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:33.957169 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:33.957176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:33.957182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:33.957382 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:33.957772 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:33.957776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:33.957778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:33.958334 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:33.958645 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:33.959100 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 08:58:33.959603 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":383,"threads_started":1}
I20260806 08:58:33.963279 5020 raft_consensus.cc:504] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } }
I20260806 08:58:33.963845 5020 leader_election.cc:302] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71af442d493417fa3ff3bd6d9c5650f; no voters:
I20260806 08:58:33.964411 5020 leader_election.cc:288] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:58:33.964521 5027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:33.965269 5027 raft_consensus.cc:686] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [term 1 LEADER]: Becoming Leader. State: Replica: e71af442d493417fa3ff3bd6d9c5650f, State: Running, Role: LEADER
I20260806 08:58:33.965770 5027 consensus_queue.cc:237] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } }
I20260806 08:58:33.966429 5020 sys_catalog.cc:564] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:58:33.968845 5030 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e71af442d493417fa3ff3bd6d9c5650f. Latest consensus state: current_term: 1 leader_uuid: "e71af442d493417fa3ff3bd6d9c5650f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } } }
I20260806 08:58:33.969274 5030 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:33.968822 5028 sys_catalog.cc:455] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e71af442d493417fa3ff3bd6d9c5650f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71af442d493417fa3ff3bd6d9c5650f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 40209 } } }
I20260806 08:58:33.969677 5028 sys_catalog.cc:458] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f [sys.catalog]: This master's current role is: LEADER
I20260806 08:58:33.971382 5034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:58:33.976044 5034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:58:33.985319 5034 catalog_manager.cc:1347] Generated new cluster ID: 751aa31b651a49d69bb0f3eca0ba4a54
I20260806 08:58:33.985445 5034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:58:33.999380 5034 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:58:34.001299 5034 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:58:34.024413 5034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: Generated new TSK 0
I20260806 08:58:34.025609 5034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:58:34.095958 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:34.443300 5054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:34.443640 5054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:34.443746 5054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:34.450977 5054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:34.451088 5054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:34.451210 5054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:58:34.461625 5054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:34.462972 5054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:34.464350 5054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:34.477912 5060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:34.477890 5061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:34.477898 5065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:34.478475 5054 server_base.cc:1034] running on GCE node
I20260806 08:58:34.479131 5054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:34.480170 5054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:34.481382 5054 hybrid_clock.cc:648] HybridClock initialized: now 1786006714481320 us; error 57 us; skew 500 ppm
I20260806 08:58:34.481707 5054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:34 dist-test-slave-jpj1 krb5kdc[4917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006714, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:34.485005 5054 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:58:34.486423 5054 webserver.cc:460] Webserver started at http://127.22.98.1:42535/ using document root <none> and password file <none>
I20260806 08:58:34.487129 5054 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:34.487262 5054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:34.487592 5054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:34.490585 5054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "c4e07b35744d41a3861177437b1c04fd"
format_stamp: "Formatted at 2026-08-06 08:58:34 on dist-test-slave-jpj1"
server_key: "ae216150f23b1bae52ab1cdf1088e46b"
server_key_iv: "2dea58feaa56b805b3cf2fe819a78f91"
server_key_version: "encryptionkey@0"
I20260806 08:58:34.491341 5054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "c4e07b35744d41a3861177437b1c04fd"
format_stamp: "Formatted at 2026-08-06 08:58:34 on dist-test-slave-jpj1"
server_key: "ae216150f23b1bae52ab1cdf1088e46b"
server_key_iv: "2dea58feaa56b805b3cf2fe819a78f91"
server_key_version: "encryptionkey@0"
I20260806 08:58:34.496814 5054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 08:58:34.500574 5077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:34.502017 5054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 08:58:34.502189 5054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "c4e07b35744d41a3861177437b1c04fd"
format_stamp: "Formatted at 2026-08-06 08:58:34 on dist-test-slave-jpj1"
server_key: "ae216150f23b1bae52ab1cdf1088e46b"
server_key_iv: "2dea58feaa56b805b3cf2fe819a78f91"
server_key_version: "encryptionkey@0"
I20260806 08:58:34.502373 5054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:34.521028 5054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:34.525490 5054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:34.525775 5054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:34.526763 5054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:34.528275 5054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:34.528349 5054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:34.528470 5054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:34.528525 5054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:34.559561 5054 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:40289
I20260806 08:58:34.559612 5228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:40289 every 8 connection(s)
I20260806 08:58:34.560793 5054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:58:34.565105 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 5054
I20260806 08:58:34.565276 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:58:34.565982 22920 external_mini_cluster.cc:1428] Setting key 840b4b7ad8113184788136f53aa2ce41
Aug 06 08:58:34 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006714, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:34.581689 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:34.563251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:58131 (local address 127.22.98.62:40209)
0806 08:58:34.563724 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:34.563729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:34.564619 (+ 890us) server_negotiation.cc:407] Connection header received
0806 08:58:34.566573 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:34.566578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:34.566667 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:34.566788 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:34.568530 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:34.569527 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:34.571105 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:34.571671 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:34.571803 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:34.575631 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:34.575638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:34.575642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:34.575681 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:34.578572 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:34.579282 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:34.579287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:34.579290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:34.579434 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:34.579892 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:34.579897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:34.579900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:34.580241 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:34.580368 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:34.581255 (+ 887us) server_negotiation.cc:299] Negotiation successful
0806 08:58:34.581437 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":234,"threads_started":1}
I20260806 08:58:34.583074 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:34.563687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40209 (local address 127.22.98.1:58131)
0806 08:58:34.564439 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:34.564485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:34.566295 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:34.567027 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:34.567036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:34.567643 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:34.568342 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:34.568360 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:34.569713 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:34.569718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:34.570945 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:34.570957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:34.571930 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:34.571935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:34.572195 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:34.572901 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:34.572931 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:34.575371 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:34.578722 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:34.578730 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:34.578744 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:34.579165 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:34.579546 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:34.579551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:34.579554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:34.579786 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:34.580363 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:34.580371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:34.580861 (+ 490us) client_negotiation.cc:769] Sending connection context
0806 08:58:34.581169 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 08:58:34.581731 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":293,"threads_started":1}
I20260806 08:58:34.584653 5230 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40209
I20260806 08:58:34.584954 5230 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:34.585681 5230 heartbeater.cc:511] Master 127.22.98.62:40209 requested a full tablet report, sending...
I20260806 08:58:34.587900 4977 ts_manager.cc:194] Registered new tserver with Master: c4e07b35744d41a3861177437b1c04fd (127.22.98.1:40289)
I20260806 08:58:34.589411 4977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:58131
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 08:58:34.630785 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:34.972688 5238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:34.972996 5238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:34.973142 5238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:34.980429 5238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:34.980561 5238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:34.980726 5238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:58:34.990608 5238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:34.991999 5238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:34.993417 5238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:35.006935 5247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:35.006999 5248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:35.007201 5250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:35.008271 5238 server_base.cc:1034] running on GCE node
I20260806 08:58:35.009007 5238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:35.010063 5238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:35.011264 5238 hybrid_clock.cc:648] HybridClock initialized: now 1786006715011202 us; error 55 us; skew 500 ppm
I20260806 08:58:35.011615 5238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:35 dist-test-slave-jpj1 krb5kdc[4917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006715, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:35.014879 5238 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:58:35.016467 5238 webserver.cc:460] Webserver started at http://127.22.98.2:33569/ using document root <none> and password file <none>
I20260806 08:58:35.017207 5238 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:35.017334 5238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:35.017637 5238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:35.020717 5238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "17fb177903844f6e84ee7ca8b71e2fce"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "4d1ce7ea5d252ad66de4a28b398334ec"
server_key_iv: "d27c3fa5d57f35949a3ef20c341ff0e1"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.021488 5238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "17fb177903844f6e84ee7ca8b71e2fce"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "4d1ce7ea5d252ad66de4a28b398334ec"
server_key_iv: "d27c3fa5d57f35949a3ef20c341ff0e1"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.027063 5238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 08:58:35.030656 5259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.032148 5238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 08:58:35.032311 5238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "17fb177903844f6e84ee7ca8b71e2fce"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "4d1ce7ea5d252ad66de4a28b398334ec"
server_key_iv: "d27c3fa5d57f35949a3ef20c341ff0e1"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.032493 5238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:35.057819 5238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:35.062587 5238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:35.062889 5238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:35.063898 5238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:35.065356 5238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:35.065439 5238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.065567 5238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:35.065694 5238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.097569 5238 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:42991
I20260806 08:58:35.097640 5409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:42991 every 8 connection(s)
I20260806 08:58:35.098835 5238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:58:35.108661 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 5238
I20260806 08:58:35.108829 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:58:35.109252 22920 external_mini_cluster.cc:1428] Setting key 6736cdc0770f00fc47ce88a113a91ec6
Aug 06 08:58:35 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006715, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:35.118409 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.101352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:49581 (local address 127.22.98.62:40209)
0806 08:58:35.101817 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.101824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.102795 (+ 971us) server_negotiation.cc:407] Connection header received
0806 08:58:35.104619 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.104624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.104697 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.104795 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:35.106286 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.107031 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.108685 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.109207 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.109330 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.112803 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:35.112809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:35.112812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:35.112846 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:35.115372 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.116123 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.116129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.116133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.116271 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.116701 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.116706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.116709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.116972 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:35.117123 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.117816 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.118044 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":211,"threads_started":1}
I20260806 08:58:35.119393 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.101914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40209 (local address 127.22.98.2:49581)
0806 08:58:35.102643 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.102681 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.104365 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.104979 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.104987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.105513 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:35.106129 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.106147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.107181 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.107185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.108478 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.108496 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.109445 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.109453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.109691 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.110308 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:35.110333 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:35.112572 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:35.115508 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:35.115517 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:35.115570 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:35.115993 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:35.116378 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:35.116383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:35.116387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:35.116583 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:35.117111 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:35.117118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:35.117439 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.117681 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.118215 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":297,"threads_started":1}
I20260806 08:58:35.120981 5411 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40209
I20260806 08:58:35.121366 5411 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:35.122186 5411 heartbeater.cc:511] Master 127.22.98.62:40209 requested a full tablet report, sending...
I20260806 08:58:35.123907 4977 ts_manager.cc:194] Registered new tserver with Master: 17fb177903844f6e84ee7ca8b71e2fce (127.22.98.2:42991)
I20260806 08:58:35.124737 4977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:49581
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:58:35.171314 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:58:35.514937 5423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:58:35.515239 5423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:58:35.515375 5423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:58:35.522753 5423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:58:35.522858 5423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:58:35.523008 5423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:58:35.532821 5423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:40209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:58:35.534121 5423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:58:35.535673 5423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:58:35.548693 5434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:35.548909 5430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:58:35.548818 5432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:58:35.549891 5423 server_base.cc:1034] running on GCE node
I20260806 08:58:35.550554 5423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:58:35.551466 5423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:58:35.552704 5423 hybrid_clock.cc:648] HybridClock initialized: now 1786006715552647 us; error 50 us; skew 500 ppm
I20260806 08:58:35.552999 5423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:58:35 dist-test-slave-jpj1 krb5kdc[4917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006715, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:58:35.556010 5423 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:58:35.557404 5423 webserver.cc:460] Webserver started at http://127.22.98.3:44439/ using document root <none> and password file <none>
I20260806 08:58:35.558089 5423 fs_manager.cc:359] Metadata directory not provided
I20260806 08:58:35.558195 5423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:58:35.558496 5423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 08:58:35.561358 5423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "0d01d2083d9548baae5315e2deaf4b81"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "049339ef543557631a51c2a1eb989703"
server_key_iv: "0a6aba204d1c28828e40d2f5384c4746"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.562073 5423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "0d01d2083d9548baae5315e2deaf4b81"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "049339ef543557631a51c2a1eb989703"
server_key_iv: "0a6aba204d1c28828e40d2f5384c4746"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.567108 5423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 08:58:35.570626 5445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.571993 5423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 08:58:35.572155 5423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "0d01d2083d9548baae5315e2deaf4b81"
format_stamp: "Formatted at 2026-08-06 08:58:35 on dist-test-slave-jpj1"
server_key: "049339ef543557631a51c2a1eb989703"
server_key_iv: "0a6aba204d1c28828e40d2f5384c4746"
server_key_version: "encryptionkey@0"
I20260806 08:58:35.572353 5423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:58:35.589825 5423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:58:35.592768 5230 heartbeater.cc:503] Master 127.22.98.62:40209 was elected leader, sending a full tablet report...
I20260806 08:58:35.594143 5423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:58:35.594424 5423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:58:35.595434 5423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:58:35.596834 5423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:58:35.596908 5423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.597003 5423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:58:35.597047 5423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:58:35.628906 5423 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:39333
I20260806 08:58:35.628954 5600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:39333 every 8 connection(s)
I20260806 08:58:35.630194 5423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:58:35.640486 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 5423
I20260806 08:58:35.640640 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:58:35.641093 22920 external_mini_cluster.cc:1428] Setting key 2eb913c57e1f7d49307be88bc1b2bd29
Aug 06 08:58:35 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006715, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:58:35.650311 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.632833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:48879 (local address 127.22.98.62:40209)
0806 08:58:35.633323 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.633328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.634116 (+ 788us) server_negotiation.cc:407] Connection header received
0806 08:58:35.636010 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.636016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.636118 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.636238 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:35.637772 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.638539 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.640428 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.640917 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.641023 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.645056 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:35.645061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:35.645065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:35.645098 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:35.647630 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.648335 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.648338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.648340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.648459 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.648831 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.648834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.648836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.649091 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:35.649202 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.649850 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.650018 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":245,"threads_started":1}
I20260806 08:58:35.651289 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.633230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:40209 (local address 127.22.98.3:48879)
0806 08:58:35.633956 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.634002 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.635718 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.636493 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.636502 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.637023 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:58:35.637594 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.637614 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.638709 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.638804 (+ 95us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.640231 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.640248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.641160 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.641170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.641466 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.642199 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:58:35.642233 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:58:35.644864 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:58:35.647771 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:35.647779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:35.647792 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:35.648237 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:35.648551 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:58:35.648554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:58:35.648556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:58:35.648751 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:58:35.649183 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:58:35.649189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:58:35.649503 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.649771 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.650162 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":288,"threads_started":1}
I20260806 08:58:35.652799 5601 heartbeater.cc:347] Connected to a master server at 127.22.98.62:40209
I20260806 08:58:35.653105 5601 heartbeater.cc:464] Registering TS with master...
I20260806 08:58:35.653873 5601 heartbeater.cc:511] Master 127.22.98.62:40209 requested a full tablet report, sending...
I20260806 08:58:35.655625 4977 ts_manager.cc:194] Registered new tserver with Master: 0d01d2083d9548baae5315e2deaf4b81 (127.22.98.3:39333)
I20260806 08:58:35.656623 4977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:48879
I20260806 08:58:35.657140 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 08:58:35.674069 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.661846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.22.98.62:40209)
0806 08:58:35.662017 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.662022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.662577 (+ 555us) server_negotiation.cc:407] Connection header received
0806 08:58:35.662738 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.662743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.662834 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.662932 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:35.664396 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.665146 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.666356 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.666818 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.666932 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.669169 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:35.669174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:35.669177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:35.669207 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:35.671632 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.672259 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.672264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.672268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.672385 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.672686 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.672688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.672690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.673047 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:35.673251 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.673562 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.673745 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 08:58:35.678447 4977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:58:35.685222 4977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:58:35.708951 5624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.697899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.22.98.1:40289)
0806 08:58:35.698715 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.698725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.699338 (+ 613us) server_negotiation.cc:407] Connection header received
0806 08:58:35.699691 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.699700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.699962 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.700126 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:35.701469 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.703119 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.706229 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.707699 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.707829 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.708053 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.708459 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.708698 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":479,"threads_started":1}
I20260806 08:58:35.709463 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.697282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39333 (local address 127.0.0.1:56808)
0806 08:58:35.699135 (+ 1853us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.699236 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.699414 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.700597 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.700605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.700636 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:35.701144 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.701161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.703327 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.703332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.706064 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.706075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.708393 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.708400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.708565 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.708597 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.708660 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.708853 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1572,"mutex_wait_us":10,"spinlock_wait_cycles":1280,"thread_start_us":1589,"threads_started":1}
I20260806 08:58:35.710194 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.697718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.22.98.3:39333)
0806 08:58:35.700002 (+ 2284us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.700011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.700042 (+ 31us) server_negotiation.cc:407] Connection header received
0806 08:58:35.700135 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.700142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.700357 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.701387 (+ 1030us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:35.701673 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.703194 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.706503 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.708160 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.709372 (+ 1212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.709625 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.709701 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.709862 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2116,"thread_start_us":2527,"threads_started":1}
I20260806 08:58:35.710371 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.697676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:40289 (local address 127.0.0.1:40914)
0806 08:58:35.699135 (+ 1459us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.699235 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.699422 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.700771 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.700774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.700801 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:35.701277 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.701288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.703312 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.703316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.706013 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.706033 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.707896 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.707903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.708133 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.708194 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.709150 (+ 956us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.709297 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":178,"thread_start_us":1024,"threads_started":1}
I20260806 08:58:35.714414 5513 tablet_service.cc:1467] Processing CreateTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a (DEFAULT_TABLE table=test-table [id=5ca48507478642308fec670c46732058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:35.714419 5143 tablet_service.cc:1467] Processing CreateTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a (DEFAULT_TABLE table=test-table [id=5ca48507478642308fec670c46732058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:35.716040 5513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfac3bcadf524bf9bafee46f3f7b974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:35.716048 5143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfac3bcadf524bf9bafee46f3f7b974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:35.716218 5626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.699816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:42991 (local address 127.0.0.1:47468)
0806 08:58:35.706859 (+ 7043us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.706881 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.708187 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.709299 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.709303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.709329 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:35.709768 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.709780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.711630 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.711635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.713589 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.713606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.715415 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.715422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.715618 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.715653 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.715763 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.715893 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6500,"thread_start_us":6340,"threads_started":1}
I20260806 08:58:35.716300 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.700086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.22.98.2:42991)
0806 08:58:35.708696 (+ 8610us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.708704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.708731 (+ 27us) server_negotiation.cc:407] Connection header received
0806 08:58:35.708821 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.708828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.709064 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.709949 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:35.710204 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.711403 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.713765 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.715156 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.715343 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.715525 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.715810 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.715981 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8445,"thread_start_us":8321,"threads_started":1}
I20260806 08:58:35.720144 5321 tablet_service.cc:1467] Processing CreateTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a (DEFAULT_TABLE table=test-table [id=5ca48507478642308fec670c46732058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:58:35.721522 5321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfac3bcadf524bf9bafee46f3f7b974a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:58:35.727561 5628 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Bootstrap starting.
I20260806 08:58:35.730130 5629 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Bootstrap starting.
I20260806 08:58:35.731200 5628 tablet_bootstrap.cc:654] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:35.732594 5628 log.cc:824] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:35.733696 5629 tablet_bootstrap.cc:654] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:35.734869 5629 log.cc:824] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:35.735203 5632 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Bootstrap starting.
I20260806 08:58:35.736399 5628 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: No bootstrap required, opened a new log
I20260806 08:58:35.736658 5628 ts_tablet_manager.cc:1397] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 08:58:35.738188 5629 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: No bootstrap required, opened a new log
I20260806 08:58:35.738463 5629 ts_tablet_manager.cc:1397] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 08:58:35.738780 5632 tablet_bootstrap.cc:654] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Neither blocks nor log segments found. Creating new log.
I20260806 08:58:35.740059 5632 log.cc:824] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Log is configured to *not* fsync() on all Append() calls
I20260806 08:58:35.744330 5632 tablet_bootstrap.cc:492] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: No bootstrap required, opened a new log
I20260806 08:58:35.744628 5632 ts_tablet_manager.cc:1397] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 08:58:35.744925 5628 raft_consensus.cc:348] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.745273 5628 raft_consensus.cc:374] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:35.745378 5628 raft_consensus.cc:729] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d01d2083d9548baae5315e2deaf4b81, State: Initialized, Role: FOLLOWER
I20260806 08:58:35.745946 5628 consensus_queue.cc:260] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.748275 5601 heartbeater.cc:503] Master 127.22.98.62:40209 was elected leader, sending a full tablet report...
I20260806 08:58:35.748565 5629 raft_consensus.cc:348] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.748965 5628 ts_tablet_manager.cc:1428] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260806 08:58:35.749009 5629 raft_consensus.cc:374] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:35.749099 5629 raft_consensus.cc:729] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e07b35744d41a3861177437b1c04fd, State: Initialized, Role: FOLLOWER
I20260806 08:58:35.749724 5629 consensus_queue.cc:260] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.751518 5629 ts_tablet_manager.cc:1428] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 08:58:35.755421 5632 raft_consensus.cc:348] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.755864 5632 raft_consensus.cc:374] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:58:35.755949 5632 raft_consensus.cc:729] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fb177903844f6e84ee7ca8b71e2fce, State: Initialized, Role: FOLLOWER
I20260806 08:58:35.756467 5632 consensus_queue.cc:260] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.757426 5411 heartbeater.cc:503] Master 127.22.98.62:40209 was elected leader, sending a full tablet report...
I20260806 08:58:35.757910 5632 ts_tablet_manager.cc:1428] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 08:58:35.806952 5637 raft_consensus.cc:482] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:58:35.807320 5637 raft_consensus.cc:504] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.809463 5637 leader_election.cc:288] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d01d2083d9548baae5315e2deaf4b81 (127.22.98.3:39333), c4e07b35744d41a3861177437b1c04fd (127.22.98.1:40289)
W20260806 08:58:35.813079 5232 tablet.cc:2369] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:58:35.818123 5624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.809935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:38329 (local address 127.22.98.1:40289)
0806 08:58:35.810178 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.810185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.811001 (+ 816us) server_negotiation.cc:407] Connection header received
0806 08:58:35.811276 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.811282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.811346 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.811423 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:35.812175 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.812955 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.815723 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.816992 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.817130 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.817243 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.817728 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.817870 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 08:58:35.818238 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.809938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:40289 (local address 127.22.98.2:38329)
0806 08:58:35.810874 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.810887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.811047 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.811459 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.811463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.811490 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:35.811998 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.812007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.813108 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.813112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.815523 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.815565 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.817335 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.817343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.817529 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.817565 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.817663 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.817827 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":72,"thread_start_us":485,"threads_started":1}
I20260806 08:58:35.819242 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.810008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:52405 (local address 127.22.98.3:39333)
0806 08:58:35.810223 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.810228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.811069 (+ 841us) server_negotiation.cc:407] Connection header received
0806 08:58:35.811168 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.811174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.811285 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.811381 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:58:35.812061 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.812803 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.815736 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.816995 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.817137 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.817243 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.818849 (+ 1606us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.818994 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 08:58:35.819615 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfac3bcadf524bf9bafee46f3f7b974a" candidate_uuid: "17fb177903844f6e84ee7ca8b71e2fce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4e07b35744d41a3861177437b1c04fd" is_pre_election: true
I20260806 08:58:35.820006 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.809822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:39333 (local address 127.22.98.2:52405)
0806 08:58:35.810848 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 08:58:35.810879 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 08:58:35.811065 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:58:35.811427 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:58:35.811432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:58:35.811462 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:58:35.811898 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.811909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.812931 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.812935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.815527 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:58:35.815565 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.818474 (+ 2909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.818481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:58:35.818655 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.818687 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 08:58:35.819441 (+ 754us) client_negotiation.cc:240] Negotiation successful
0806 08:58:35.819618 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":163,"spinlock_wait_cycles":48896,"thread_start_us":570,"threads_started":1}
I20260806 08:58:35.820336 5165 raft_consensus.cc:2454] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb177903844f6e84ee7ca8b71e2fce in term 0.
I20260806 08:58:35.821106 5264 leader_election.cc:302] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fb177903844f6e84ee7ca8b71e2fce, c4e07b35744d41a3861177437b1c04fd; no voters:
I20260806 08:58:35.821065 5539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfac3bcadf524bf9bafee46f3f7b974a" candidate_uuid: "17fb177903844f6e84ee7ca8b71e2fce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d01d2083d9548baae5315e2deaf4b81" is_pre_election: true
I20260806 08:58:35.821547 5637 raft_consensus.cc:2781] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:58:35.821635 5539 raft_consensus.cc:2454] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb177903844f6e84ee7ca8b71e2fce in term 0.
I20260806 08:58:35.821738 5637 raft_consensus.cc:482] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:58:35.821853 5637 raft_consensus.cc:3037] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:35.824262 5637 raft_consensus.cc:504] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.824893 5637 leader_election.cc:288] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [CANDIDATE]: Term 1 election: Requested vote from peers 0d01d2083d9548baae5315e2deaf4b81 (127.22.98.3:39333), c4e07b35744d41a3861177437b1c04fd (127.22.98.1:40289)
I20260806 08:58:35.825347 5539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfac3bcadf524bf9bafee46f3f7b974a" candidate_uuid: "17fb177903844f6e84ee7ca8b71e2fce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d01d2083d9548baae5315e2deaf4b81"
I20260806 08:58:35.825543 5539 raft_consensus.cc:3037] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:35.825621 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfac3bcadf524bf9bafee46f3f7b974a" candidate_uuid: "17fb177903844f6e84ee7ca8b71e2fce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4e07b35744d41a3861177437b1c04fd"
I20260806 08:58:35.825821 5165 raft_consensus.cc:3037] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:58:35.828096 5539 raft_consensus.cc:2454] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb177903844f6e84ee7ca8b71e2fce in term 1.
I20260806 08:58:35.828248 5165 raft_consensus.cc:2454] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb177903844f6e84ee7ca8b71e2fce in term 1.
I20260806 08:58:35.828619 5262 leader_election.cc:302] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d01d2083d9548baae5315e2deaf4b81, 17fb177903844f6e84ee7ca8b71e2fce; no voters:
I20260806 08:58:35.829007 5637 raft_consensus.cc:2781] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:58:35.829515 5637 raft_consensus.cc:686] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 1 LEADER]: Becoming Leader. State: Replica: 17fb177903844f6e84ee7ca8b71e2fce, State: Running, Role: LEADER
I20260806 08:58:35.829989 5637 consensus_queue.cc:237] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } }
I20260806 08:58:35.835037 4977 catalog_manager.cc:5495] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17fb177903844f6e84ee7ca8b71e2fce (127.22.98.2). New cstate: current_term: 1 leader_uuid: "17fb177903844f6e84ee7ca8b71e2fce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17fb177903844f6e84ee7ca8b71e2fce" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 42991 } health_report { overall_health: HEALTHY } } }
W20260806 08:58:35.850862 5413 tablet.cc:2369] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:58:35.870289 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.864819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.22.98.2:42991)
0806 08:58:35.865000 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.865005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.865088 (+ 83us) server_negotiation.cc:407] Connection header received
0806 08:58:35.865278 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.865281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.865360 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.865493 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:58:35.865841 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.866626 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.868136 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.868550 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.868664 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.869179 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:58:35.869736 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:58:35.869870 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.869987 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.870057 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 08:58:35.879029 5165 raft_consensus.cc:1261] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 1 FOLLOWER]: Refusing update from remote peer 17fb177903844f6e84ee7ca8b71e2fce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:35.879033 5539 raft_consensus.cc:1261] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 1 FOLLOWER]: Refusing update from remote peer 17fb177903844f6e84ee7ca8b71e2fce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:58:35.880031 5640 consensus_queue.cc:1035] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d01d2083d9548baae5315e2deaf4b81" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 39333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:58:35.880355 5637 consensus_queue.cc:1035] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4e07b35744d41a3861177437b1c04fd" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 08:58:35.881791 5602 tablet.cc:2369] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:58:35.912971 4977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:35.913296 4977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:58:35.917644 4977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e71af442d493417fa3ff3bd6d9c5650f: Sending DeleteTablet for 3 replicas of tablet dfac3bcadf524bf9bafee46f3f7b974a
I20260806 08:58:35.918891 5321 tablet_service.cc:1514] Processing DeleteTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:35 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:47468
I20260806 08:58:35.918895 5513 tablet_service.cc:1514] Processing DeleteTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:35 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:56808
I20260806 08:58:35.919693 5143 tablet_service.cc:1514] Processing DeleteTablet for tablet dfac3bcadf524bf9bafee46f3f7b974a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:58:35 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:40914
I20260806 08:58:35.920490 5660 tablet_replica.cc:330] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: stopping tablet replica
I20260806 08:58:35.920730 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 5054
I20260806 08:58:35.920974 5660 raft_consensus.cc:2229] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:35.921669 5660 raft_consensus.cc:2258] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:35.924503 5660 ts_tablet_manager.cc:1905] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:35.933463 5659 tablet_replica.cc:330] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: stopping tablet replica
I20260806 08:58:35.937053 5659 raft_consensus.cc:2229] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:58:35.937896 5659 raft_consensus.cc:2258] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:35.942164 5659 ts_tablet_manager.cc:1905] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:35.942746 5660 ts_tablet_manager.cc:1918] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:35.942945 5660 log.cc:1196] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/dfac3bcadf524bf9bafee46f3f7b974a
I20260806 08:58:35.943483 5660 ts_tablet_manager.cc:1939] T dfac3bcadf524bf9bafee46f3f7b974a P c4e07b35744d41a3861177437b1c04fd: Deleting consensus metadata
I20260806 08:58:35.945636 4960 catalog_manager.cc:4841] TS c4e07b35744d41a3861177437b1c04fd (127.22.98.1:40289): tablet dfac3bcadf524bf9bafee46f3f7b974a (table test-table [id=5ca48507478642308fec670c46732058]) successfully deleted
Aug 06 08:58:35 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:58:35.953887 5659 ts_tablet_manager.cc:1918] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:35.954660 5658 tablet_replica.cc:330] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: stopping tablet replica
I20260806 08:58:35.955128 5658 raft_consensus.cc:2229] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:58:35.955729 5658 raft_consensus.cc:2258] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:58:35.957512 5659 log.cc:1196] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/dfac3bcadf524bf9bafee46f3f7b974a
I20260806 08:58:35.958098 5659 ts_tablet_manager.cc:1939] T dfac3bcadf524bf9bafee46f3f7b974a P 17fb177903844f6e84ee7ca8b71e2fce: Deleting consensus metadata
I20260806 08:58:35.960199 5658 ts_tablet_manager.cc:1905] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:58:35.965801 5624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:35.921588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.22.98.1:40289)
0806 08:58:35.921768 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:35.921774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:35.928468 (+ 6694us) server_negotiation.cc:407] Connection header received
0806 08:58:35.928669 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:35.928675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:35.928757 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:35.928868 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:35.933785 (+ 4917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.934894 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.938429 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:35.939073 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:35.939175 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:35.957461 (+ 18286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:35.957484 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:35.957505 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:35.957566 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:35.962025 (+ 4459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.963078 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.963090 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.963093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.963335 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:35.963916 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:35.963923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:35.963927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:35.964465 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:35.964820 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:35.965161 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 08:58:35.965410 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 08:58:35.965991 5658 ts_tablet_manager.cc:1918] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:58:35.966154 5658 log.cc:1196] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/dfac3bcadf524bf9bafee46f3f7b974a
I20260806 08:58:35.966531 5223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:35.966666 5658 ts_tablet_manager.cc:1939] T dfac3bcadf524bf9bafee46f3f7b974a P 0d01d2083d9548baae5315e2deaf4b81: Deleting consensus metadata
I20260806 08:58:35.969027 4957 catalog_manager.cc:4841] TS 17fb177903844f6e84ee7ca8b71e2fce (127.22.98.2:42991): tablet dfac3bcadf524bf9bafee46f3f7b974a (table test-table [id=5ca48507478642308fec670c46732058]) successfully deleted
I20260806 08:58:35.970007 4958 catalog_manager.cc:4841] TS 0d01d2083d9548baae5315e2deaf4b81 (127.22.98.3:39333): tablet dfac3bcadf524bf9bafee46f3f7b974a (table test-table [id=5ca48507478642308fec670c46732058]) successfully deleted
I20260806 08:58:36.057042 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 5054
I20260806 08:58:36.089207 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 5238
Aug 06 08:58:36 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:58:36.109548 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:36.090194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.22.98.2:42991)
0806 08:58:36.090364 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:36.090368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:36.090388 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:58:36.090599 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:36.090604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:36.090678 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:36.090771 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:36.092507 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:36.093451 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:36.095413 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:36.096036 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:36.096159 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:36.101802 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:36.101823 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:36.101841 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:36.101898 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:36.106209 (+ 4311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:36.107114 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:36.107125 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:36.107129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:36.107317 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:36.107846 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:36.107851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:36.107856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:36.108321 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:36.108685 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:36.108933 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 08:58:36.109163 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 08:58:36.110273 5405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:36.227679 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 5238
I20260806 08:58:36.253159 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 5423
Aug 06 08:58:36 dist-test-slave-jpj1 krb5kdc[4917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:58:36.269205 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:58:36.254193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.22.98.3:39333)
0806 08:58:36.254358 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 08:58:36.254362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:58:36.254383 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:58:36.254595 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:58:36.254600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:58:36.254676 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:58:36.254783 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:58:36.256461 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:36.257181 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:36.258530 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:58:36.258986 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:58:36.259107 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:58:36.263455 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:58:36.263473 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:58:36.263492 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:58:36.263558 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:58:36.266384 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:36.267081 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:36.267089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:36.267092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:36.267297 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:58:36.267766 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:58:36.267771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:58:36.267773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:58:36.268181 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:58:36.268431 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 08:58:36.268753 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 08:58:36.268933 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 08:58:36.269879 5594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:36.360005 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 5423
I20260806 08:58:36.383322 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 4938
I20260806 08:58:36.384783 5015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:58:36.495406 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 4938
2026-08-06T08:58:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4513 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:58:36 dist-test-slave-jpj1 krb5kdc[5669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:58:36 dist-test-slave-jpj1 krb5kdc[5669](info): set up 2 sockets
Aug 06 08:58:36 dist-test-slave-jpj1 krb5kdc[5669](info): commencing operation
krb5kdc: starting...
W20260806 08:58:37.698719 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.003s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:58:37 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:58:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangeradmin_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangeradmin_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangerlookup_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangerlookup_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
I20260806 08:58:38.118322 22920 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6BBFDc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/postgres -l logfile start
2026-08-06 08:58:40.532 UTC [5715] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 08:58:40.532 UTC [5715] LOG: listening on IPv4 address "127.22.98.20", port 53715
2026-08-06 08:58:40.536 UTC [5715] LOG: listening on Unix socket "/tmp/.s.PGSQL.53715"
2026-08-06 08:58:40.560 UTC [5718] LOG: database system was shut down at 2026-08-06 08:58:39 UTC
2026-08-06 08:58:40.565 UTC [5715] LOG: database system is ready to accept connections
I20260806 08:58:41.624058 22920 mini_postgres.cc:96] Postgres bound to 53715
2026-08-06 08:58:41.649 UTC [5729] FATAL: database "slave" does not exist
127.22.98.20:53715 - accepting connections
I20260806 08:58:41.650121 22920 mini_ranger.cc:161] Starting Ranger...
I20260806 08:58:41.683451 22920 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 08:58:42.135052 22920 mini_ranger.cc:87] Created ranger Postgres database
I20260806 08:58:42.135215 22920 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin
2026-08-06 08:58:42,480 [I] DB FLAVOR :POSTGRES
2026-08-06 08:58:42,480 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 08:58:42,480 [I] Checking connection..
2026-08-06 08:58:42,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 08:58:42,754 [I] Checking connection passed.
2026-08-06 08:58:42,754 [I] --------- Verifying version history table ---------
2026-08-06 08:58:42,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:58:43,022 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 08:58:43,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:58:43,272 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 08:58:43,272 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 08:58:43,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 08:58:43.520 UTC [5816] WARNING: there is no transaction in progress
2026-08-06 08:58:43,528 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 08:58:43,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:58:43,797 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 08:58:43,797 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 08:58:43,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 08:58:44,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 08:58:44,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jpj1.c.gcp-upstream.internal','N') ;"
2026-08-06 08:58:44,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 08:58:44,776 [I] Table x_portal_user does not exist in database ranger
2026-08-06 08:58:44,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 08:58:45,023 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 08:58:45,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 08:58:45,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 08:58:45,549 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 08:58:45,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 08:58:46.502 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.516 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.532 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.546 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.560 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.576 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.629 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.635 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.649 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.659 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.668 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.681 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.690 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.697 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.703 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:46.709 UTC [5973] WARNING: there is no transaction in progress
2026-08-06 08:58:47,328 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 08:58:47,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 08:58:47,579 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 08:58:47,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 08:58:47,845 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 08:58:47,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 08:58:48,113 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 08:58:48,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 08:58:48,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 08:58:48,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jpj1.c.gcp-upstream.internal';"
2026-08-06 08:58:48,902 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 08:58:48,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 08:58:49,164 [I] DB_PATCHES have already been applied
I20260806 08:58:49.170553 22920 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6BBFDc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 08:58:49.170693 22920 mini_ranger.cc:193] Using host: 127.22.98.20
I20260806 08:58:49.192196 22920 mini_ranger.cc:227] Ranger admin URL: http://127.22.98.20:43493
Aug 06, 2026 8:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 8:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 8:58:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 8:58:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 8:58:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 08:58:50.410351 22920 mini_ranger.cc:160] Time spent starting Ranger: real 8.760s user 0.000s sys 0.068s
Aug 06, 2026 8:59:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 8:59:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 8:59:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 8:59:10 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 8:59:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 08:59:11.818342 22920 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangerkms_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/rangerkms_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.98.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.98.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 08:59:12 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006752, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 08:59:12.023303 22920 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 08:59:12.051280 22920 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 08:59:12.917840 22920 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 08:59:12.917992 22920 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms
I20260806 08:59:12.918151 22920 mini_ranger_kms.cc:227] Using postgres at 127.22.98.20:53715
2026-08-06 08:59:13,614 [I] DB FLAVOR :POSTGRES
2026-08-06 08:59:13,614 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 08:59:13,614 [I] Checking connection
2026-08-06 08:59:13,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 08:59:13,873 [I] connection success
2026-08-06 08:59:13,874 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 08:59:13,874 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 08:59:13,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 08:59:14,139 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 08:59:14,139 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 08:59:14,139 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 08:59:14,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:53715/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 08:59:14,430 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 08:59:14.945305 22920 mini_ranger_kms.cc:310] Created kms service
I20260806 08:59:15.146297 22920 mini_ranger_kms.cc:326] Created kudu user
I20260806 08:59:15.190267 22920 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 08:59:15 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006752, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:15.594923 22920 mini_ranger_kms.cc:384] Added ranger policy
I20260806 08:59:15.595311 22920 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms:/tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 08:59:15.595366 22920 mini_ranger_kms.cc:240] Using host: 127.22.98.20
I20260806 08:59:15.620535 22920 mini_ranger_kms.cc:276] Ranger KMS PID: 6206
I20260806 08:59:15.620726 22920 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.98.20:58249
Aug 06, 2026 8:59:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:59:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:59:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 8:59:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 8:59:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 8:59:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 08:59:18 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786006758, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:59:18.104578 22920 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.081s user 0.166s sys 0.435s
I20260806 08:59:18.105085 22920 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 08:59:18.105340 22920 mini_ranger_kms.cc:401] 127.22.98.20:58249/kms/v1/keys
Aug 06 08:59:18 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006758, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
Aug 06 08:59:18 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006758, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
Aug 06 08:59:21 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 08:59:21.464401 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37091
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37091
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:58249/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:59:21.948292 6273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:59:21.948654 6273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:59:21.948792 6273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:59:21.957440 6273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 08:59:21.957556 6273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:59:21.957633 6273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:59:21.957711 6273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 08:59:21.957779 6273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 08:59:21.970280 6273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.98.20:58249/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37091
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:59:21.971720 6273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:59:21.973137 6273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:59:21.988209 6281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:21.988327 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:21.988324 6283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:59:21.990476 6273 server_base.cc:1034] running on GCE node
I20260806 08:59:21.991389 6273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:59:21.993063 6273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:59:21.994341 6273 hybrid_clock.cc:648] HybridClock initialized: now 1786006761994231 us; error 37 us; skew 500 ppm
I20260806 08:59:21.994800 6273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:59:21 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:59:21.998278 6273 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 08:59:22.000020 6273 webserver.cc:460] Webserver started at http://127.22.98.62:36009/ using document root <none> and password file <none>
I20260806 08:59:22.000898 6273 fs_manager.cc:359] Metadata directory not provided
I20260806 08:59:22.001053 6273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:59:22.001458 6273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 08:59:21 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:22.066015 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "cfe5c112b3c444b6961ee9112cd4f42f"
format_stamp: "Formatted at 2026-08-06 08:59:22 on dist-test-slave-jpj1"
server_key: "048aea1b47a2ebc95a5c01d47c90bceb"
server_key_iv: "0f2fbf935e28c66481d6c587457c5872"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:22.067619 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "cfe5c112b3c444b6961ee9112cd4f42f"
format_stamp: "Formatted at 2026-08-06 08:59:22 on dist-test-slave-jpj1"
server_key: "048aea1b47a2ebc95a5c01d47c90bceb"
server_key_iv: "0f2fbf935e28c66481d6c587457c5872"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:22.076869 6273 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.003s
I20260806 08:59:22.101110 6296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:22.103111 6273 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 08:59:22.103436 6273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "cfe5c112b3c444b6961ee9112cd4f42f"
format_stamp: "Formatted at 2026-08-06 08:59:22 on dist-test-slave-jpj1"
server_key: "048aea1b47a2ebc95a5c01d47c90bceb"
server_key_iv: "0f2fbf935e28c66481d6c587457c5872"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:22.103937 6273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:59:22.124130 6273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:59:22.130015 6273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:59:22.130509 6273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:59:22.163638 6273 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37091
I20260806 08:59:22.163812 6370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37091 every 8 connection(s)
I20260806 08:59:22.166111 6273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 08:59:22.170864 6371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:59:22.175483 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 6273
I20260806 08:59:22.175740 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 08:59:22.176586 22920 external_mini_cluster.cc:1428] Setting key 2ea0c0316d88c1e370762bfe56ba96c1
I20260806 08:59:22.187412 6371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: Bootstrap starting.
I20260806 08:59:22.193533 6371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: Neither blocks nor log segments found. Creating new log.
I20260806 08:59:22.195989 6371 log.cc:824] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: Log is configured to *not* fsync() on all Append() calls
I20260806 08:59:22.200459 6371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: No bootstrap required, opened a new log
Aug 06 08:59:22 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:59:22.213833 6371 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } }
I20260806 08:59:22.214397 6371 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:59:22.214530 6371 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe5c112b3c444b6961ee9112cd4f42f, State: Initialized, Role: FOLLOWER
I20260806 08:59:22.215329 6371 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } }
I20260806 08:59:22.215817 6371 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 08:59:22.216039 6371 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 08:59:22.216363 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:22.178984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.22.98.62:37091)
0806 08:59:22.193096 (+ 14112us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:22.193123 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:22.193435 (+ 312us) server_negotiation.cc:407] Connection header received
0806 08:59:22.196526 (+ 3091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:22.196565 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:22.197194 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:22.197616 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:22.199316 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:22.200953 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:22.202429 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:22.203221 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:22.203428 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:22.207441 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:22.207455 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:22.207475 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:22.207522 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:22.211286 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:22.212151 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:22.212158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:22.212166 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:22.212382 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:22.212859 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:22.212863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:22.212866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:22.213520 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:22.213811 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:22.214454 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 08:59:22.215000 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13771,"thread_start_us":14034,"threads_started":1}
I20260806 08:59:22.216356 6371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:59:22.220906 6371 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } }
I20260806 08:59:22.221616 6371 leader_election.cc:302] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfe5c112b3c444b6961ee9112cd4f42f; no voters:
I20260806 08:59:22.222703 6371 leader_election.cc:288] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 08:59:22.222783 6377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:59:22.225735 6371 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 08:59:22.230749 6377 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [term 1 LEADER]: Becoming Leader. State: Replica: cfe5c112b3c444b6961ee9112cd4f42f, State: Running, Role: LEADER
I20260806 08:59:22.232005 6377 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } }
I20260806 08:59:22.237517 6381 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfe5c112b3c444b6961ee9112cd4f42f. Latest consensus state: current_term: 1 leader_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } } }
I20260806 08:59:22.237551 6380 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe5c112b3c444b6961ee9112cd4f42f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37091 } } }
I20260806 08:59:22.238257 6381 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [sys.catalog]: This master's current role is: LEADER
I20260806 08:59:22.238346 6380 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f [sys.catalog]: This master's current role is: LEADER
I20260806 08:59:22.241952 6386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 08:59:22.253568 6386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 08:59:22.275257 6371 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 08:59:22.284757 6386 catalog_manager.cc:1347] Generated new cluster ID: 3baf845b306b4b4fae4b3326e1aa708b
I20260806 08:59:22.285341 6386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 08:59:22.314448 6386 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 08:59:22.317127 6386 catalog_manager.cc:1504] Loading token signing keys...
I20260806 08:59:22.339391 6386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: Generated new TSK 0
I20260806 08:59:22.340785 6386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 08:59:23.198076 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:58249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 08:59:23 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 08:59:23.675463 6430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:59:23.676040 6430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:59:23.676226 6430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:59:23.688386 6430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:59:23.688548 6430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:59:23.688823 6430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 08:59:23.705248 6430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.98.20:58249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:59:23.707357 6430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:59:23.709295 6430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:59:23.726788 6442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:23.727035 6445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:23.732992 6447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:59:23.733660 6430 server_base.cc:1034] running on GCE node
I20260806 08:59:23.734365 6430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:59:23.735505 6430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:59:23.736802 6430 hybrid_clock.cc:648] HybridClock initialized: now 1786006763736727 us; error 66 us; skew 500 ppm
I20260806 08:59:23.737154 6430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:59:23 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:59:23.741086 6430 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 08:59:23.742977 6430 webserver.cc:460] Webserver started at http://127.22.98.1:37421/ using document root <none> and password file <none>
I20260806 08:59:23.743829 6430 fs_manager.cc:359] Metadata directory not provided
I20260806 08:59:23.744011 6430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:59:23.744516 6430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 08:59:23 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:23.768083 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "9945a44777ca414aae1148f8fce586da"
format_stamp: "Formatted at 2026-08-06 08:59:23 on dist-test-slave-jpj1"
server_key: "1d458a15ab1939b8d762fd4cb4c754d4"
server_key_iv: "7d49e1e8c0683b52eb4794be7aa98f2a"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:23.769896 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "9945a44777ca414aae1148f8fce586da"
format_stamp: "Formatted at 2026-08-06 08:59:23 on dist-test-slave-jpj1"
server_key: "1d458a15ab1939b8d762fd4cb4c754d4"
server_key_iv: "7d49e1e8c0683b52eb4794be7aa98f2a"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:23.784658 6430 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.008s sys 0.004s
I20260806 08:59:23.812462 6462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:23.814616 6430 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.008s sys 0.000s
I20260806 08:59:23.815062 6430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "9945a44777ca414aae1148f8fce586da"
format_stamp: "Formatted at 2026-08-06 08:59:23 on dist-test-slave-jpj1"
server_key: "1d458a15ab1939b8d762fd4cb4c754d4"
server_key_iv: "7d49e1e8c0683b52eb4794be7aa98f2a"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:23.815519 6430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:59:23.852295 6430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:59:23.858539 6430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:59:23.858937 6430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:59:23.860414 6430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:59:23.862414 6430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:59:23.862633 6430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:23.862836 6430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:59:23.862936 6430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 08:59:23 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
Aug 06 08:59:24 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:24.377262 6626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:37927 every 8 connection(s)
I20260806 08:59:24.377548 6430 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:37927
I20260806 08:59:24.379514 6430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 08:59:24.380426 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 6430
I20260806 08:59:24.380625 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 08:59:24.381418 22920 external_mini_cluster.cc:1428] Setting key 376fa03f81331392fd48d7669eed7efe
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Aug 06 08:59:24 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006763, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:59:24.477945 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:24.399831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:40329 (local address 127.22.98.62:37091)
0806 08:59:24.412010 (+ 12179us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:24.412017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:24.412042 (+ 25us) server_negotiation.cc:407] Connection header received
0806 08:59:24.421559 (+ 9517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:24.421566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:24.421778 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:24.421874 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:24.440995 (+ 19121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:24.442237 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:24.448966 (+ 6729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:24.449785 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:24.449908 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:24.465697 (+ 15789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:24.465707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:24.465714 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:24.465793 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:24.470235 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:24.471950 (+ 1715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:24.471958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:24.471962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:24.472183 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:24.475457 (+ 3274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:24.475464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:24.475468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:24.476010 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:24.476157 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:24.477299 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0806 08:59:24.477572 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12019,"thread_start_us":11856,"threads_started":1}
I20260806 08:59:24.480629 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:24.399133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37091 (local address 127.22.98.1:40329)
0806 08:59:24.410280 (+ 11147us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:24.410354 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:24.419215 (+ 8861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:24.422565 (+ 3350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:24.422578 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:24.423343 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:59:24.425234 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:24.425270 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:24.442575 (+ 17305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:24.442582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:24.448657 (+ 6075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:24.448686 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:24.450642 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:24.450653 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:24.451489 (+ 836us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:24.454900 (+ 3411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:59:24.454940 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:59:24.460779 (+ 5839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:59:24.470541 (+ 9762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:24.470554 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:24.470578 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:24.471386 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:24.472381 (+ 995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:24.472386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:24.472390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:24.472752 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:24.476314 (+ 3562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:59:24.476324 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:59:24.476828 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 08:59:24.478545 (+ 1717us) client_negotiation.cc:240] Negotiation successful
0806 08:59:24.479147 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10713,"thread_start_us":10405,"threads_started":1}
I20260806 08:59:24.482894 6627 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37091
I20260806 08:59:24.483294 6627 heartbeater.cc:464] Registering TS with master...
I20260806 08:59:24.484414 6627 heartbeater.cc:511] Master 127.22.98.62:37091 requested a full tablet report, sending...
I20260806 08:59:24.487221 6324 ts_manager.cc:194] Registered new tserver with Master: 9945a44777ca414aae1148f8fce586da (127.22.98.1:37927)
I20260806 08:59:24.489275 6324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:40329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
W20260806 08:59:24.506827 22920 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.98.2: real 0.121s user 0.001s sys 0.025s
I20260806 08:59:24.507946 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:58249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 08:59:24.950834 6637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:59:24.951134 6637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:59:24.951251 6637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:59:24.958633 6637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:59:24.958858 6637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:59:24.959033 6637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 08:59:24.969295 6637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.98.20:58249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:59:24.970729 6637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:59:24.972201 6637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:59:24.988408 6655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:24.988449 6662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:24.988371 6656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:59:24.990190 6637 server_base.cc:1034] running on GCE node
I20260806 08:59:24.991021 6637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:59:24.992369 6637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:59:24.993669 6637 hybrid_clock.cc:648] HybridClock initialized: now 1786006764993550 us; error 62 us; skew 500 ppm
I20260806 08:59:24.994088 6637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:59:24 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006764, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:59:24.997731 6637 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 08:59:24.999583 6637 webserver.cc:460] Webserver started at http://127.22.98.2:43657/ using document root <none> and password file <none>
I20260806 08:59:25.000553 6637 fs_manager.cc:359] Metadata directory not provided
I20260806 08:59:25.000697 6637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:59:25.001065 6637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 08:59:25 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006764, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:25.021569 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "b8e7f51db1944b1b8737c4da81ad8d98"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "733dddf7c7f312cea1eb666cd6685aea"
server_key_iv: "8faa384198086d5479097ffee0d696d5"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.022898 6637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "b8e7f51db1944b1b8737c4da81ad8d98"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "733dddf7c7f312cea1eb666cd6685aea"
server_key_iv: "8faa384198086d5479097ffee0d696d5"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.031469 6637 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.003s
I20260806 08:59:25.052230 6672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.054178 6637 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 08:59:25.054394 6637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "b8e7f51db1944b1b8737c4da81ad8d98"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "733dddf7c7f312cea1eb666cd6685aea"
server_key_iv: "8faa384198086d5479097ffee0d696d5"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.054693 6637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:59:25.088439 6637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:59:25.094238 6637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:59:25.094622 6637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:59:25.096038 6637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:59:25.098006 6637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:59:25.098115 6637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.098240 6637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:59:25.098311 6637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.136669 6637 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:45405
I20260806 08:59:25.136708 6833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:45405 every 8 connection(s)
I20260806 08:59:25.137950 6637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 08:59:25.142100 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 6637
I20260806 08:59:25.142285 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 08:59:25.142803 22920 external_mini_cluster.cc:1428] Setting key 5917f7ddedd938e48bc14c46fc4270c0
Aug 06 08:59:25 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006764, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:59:25.159933 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.140643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:55937 (local address 127.22.98.62:37091)
0806 08:59:25.141086 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.141091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.142072 (+ 981us) server_negotiation.cc:407] Connection header received
0806 08:59:25.144334 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.144339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.144439 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.144533 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:25.146242 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.147051 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.148877 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.149280 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.149403 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.153027 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:25.153033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:25.153036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:25.153081 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:25.155857 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.156822 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.156827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.156830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.157042 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.157718 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.157723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.157756 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.158157 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:25.158333 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.159245 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.159465 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":211,"threads_started":1}
I20260806 08:59:25.161260 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.141175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37091 (local address 127.22.98.2:55937)
0806 08:59:25.141908 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.141953 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.144041 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.144842 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.144854 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.145371 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:59:25.146054 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.146075 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.147209 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.147214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.148722 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.148736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.149515 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.149523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.149769 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.150434 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:59:25.150458 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:59:25.152784 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:59:25.156006 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:25.156017 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:25.156037 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:25.156664 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:25.157157 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:25.157161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:25.157165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:25.157467 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:25.158307 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:59:25.158316 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:59:25.158749 (+ 433us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.159079 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.159667 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":286,"threads_started":1}
I20260806 08:59:25.162753 6836 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37091
I20260806 08:59:25.163127 6836 heartbeater.cc:464] Registering TS with master...
I20260806 08:59:25.164129 6836 heartbeater.cc:511] Master 127.22.98.62:37091 requested a full tablet report, sending...
I20260806 08:59:25.165959 6324 ts_manager.cc:194] Registered new tserver with Master: b8e7f51db1944b1b8737c4da81ad8d98 (127.22.98.2:45405)
I20260806 08:59:25.167194 6324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:55937
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 08:59:25.212074 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:58249/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 08:59:25.493584 6627 heartbeater.cc:503] Master 127.22.98.62:37091 was elected leader, sending a full tablet report...
W20260806 08:59:25.582820 6845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 08:59:25.583132 6845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 08:59:25.583230 6845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 08:59:25.591385 6845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 08:59:25.591512 6845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 08:59:25.591739 6845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 08:59:25.602257 6845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.98.20:58249/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 08:59:25.603603 6845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 08:59:25.604916 6845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 08:59:25.618474 6853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:25.618505 6859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 08:59:25.618435 6854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 08:59:25.619303 6845 server_base.cc:1034] running on GCE node
I20260806 08:59:25.620007 6845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 08:59:25.621016 6845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 08:59:25.622237 6845 hybrid_clock.cc:648] HybridClock initialized: now 1786006765622176 us; error 51 us; skew 500 ppm
I20260806 08:59:25.622601 6845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 08:59:25 dist-test-slave-jpj1 krb5kdc[5669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006765, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 08:59:25.626413 6845 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 08:59:25.628052 6845 webserver.cc:460] Webserver started at http://127.22.98.3:41595/ using document root <none> and password file <none>
I20260806 08:59:25.628749 6845 fs_manager.cc:359] Metadata directory not provided
I20260806 08:59:25.628862 6845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 08:59:25.629170 6845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 08:59:25 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006765, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 08:59:25.643146 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "f9729e47166c443fbbe20872ee77630c"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "460dd66dd6b8d9e5f4e8fe20d7ae7a80"
server_key_iv: "fbc5b524618a002bdf70b24e49833692"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.644052 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "f9729e47166c443fbbe20872ee77630c"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "460dd66dd6b8d9e5f4e8fe20d7ae7a80"
server_key_iv: "fbc5b524618a002bdf70b24e49833692"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.649932 6845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 08:59:25.666934 6868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.668576 6845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 08:59:25.668772 6845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "f9729e47166c443fbbe20872ee77630c"
format_stamp: "Formatted at 2026-08-06 08:59:25 on dist-test-slave-jpj1"
server_key: "460dd66dd6b8d9e5f4e8fe20d7ae7a80"
server_key_iv: "fbc5b524618a002bdf70b24e49833692"
server_key_version: "kuduclusterkey@0"
I20260806 08:59:25.669009 6845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 08:59:25.689064 6845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 08:59:25.693012 6845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 08:59:25.693331 6845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 08:59:25.694394 6845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 08:59:25.695891 6845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 08:59:25.695976 6845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.696106 6845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 08:59:25.696161 6845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 08:59:25.730250 6845 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:46039
I20260806 08:59:25.730304 7032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:46039 every 8 connection(s)
I20260806 08:59:25.731596 6845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 08:59:25.732234 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 6845
I20260806 08:59:25.732425 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 08:59:25.732998 22920 external_mini_cluster.cc:1428] Setting key 6c27fc47fc92f3cfdec2d40afd8450aa
Aug 06 08:59:25 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006765, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 08:59:25.752058 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.734220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:51383 (local address 127.22.98.62:37091)
0806 08:59:25.734708 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.734712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.735954 (+ 1242us) server_negotiation.cc:407] Connection header received
0806 08:59:25.737969 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.737975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.738079 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.738176 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:25.739753 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.740471 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.742023 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.742418 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.742526 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.745893 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:25.745899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:25.745903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:25.745945 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:25.749119 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.749930 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.749934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.749937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.750080 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.750453 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.750456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.750458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.750779 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:25.750892 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.751527 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.751783 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":267,"threads_started":1}
I20260806 08:59:25.753067 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.734864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37091 (local address 127.22.98.3:51383)
0806 08:59:25.735809 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.735847 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.737699 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.738439 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.738447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.738885 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 08:59:25.739576 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.739607 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.740631 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.740635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.741876 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.741897 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.742637 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.742644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.742862 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.743499 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 08:59:25.743522 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 08:59:25.745658 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 08:59:25.749259 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:25.749269 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:25.749289 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:25.749808 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:25.750161 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 08:59:25.750164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 08:59:25.750166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 08:59:25.750369 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 08:59:25.750873 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 08:59:25.750879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 08:59:25.751198 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.751430 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.751909 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":463,"threads_started":1}
I20260806 08:59:25.754562 7033 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37091
I20260806 08:59:25.754966 7033 heartbeater.cc:464] Registering TS with master...
I20260806 08:59:25.755769 7033 heartbeater.cc:511] Master 127.22.98.62:37091 requested a full tablet report, sending...
I20260806 08:59:25.757304 6324 ts_manager.cc:194] Registered new tserver with Master: f9729e47166c443fbbe20872ee77630c (127.22.98.3:46039)
I20260806 08:59:25.758152 6324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:51383
I20260806 08:59:25.761832 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 08:59:25.779924 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.766944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.22.98.62:37091)
0806 08:59:25.767109 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.767114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.767528 (+ 414us) server_negotiation.cc:407] Connection header received
0806 08:59:25.767709 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.767713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.767799 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.767880 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:25.769313 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.770128 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.771906 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.772334 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.772438 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.774921 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:25.774927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:25.774930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:25.774962 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:25.777416 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.778088 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.778092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.778094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.778252 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:25.778590 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:25.778593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:25.778595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:25.778866 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:25.779046 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.779433 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.779646 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 08:59:25.784492 6324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 08:59:25.792096 6324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 08:59:25.819444 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.805591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:45405 (local address 127.0.0.1:42720)
0806 08:59:25.807293 (+ 1702us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.807346 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.807570 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.811996 (+ 4426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.812001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.812027 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:59:25.812416 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.812430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.814465 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.814469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.816612 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.816628 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.818267 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.818274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.818528 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.818597 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.818709 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.818927 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"mutex_wait_us":58,"thread_start_us":2385,"threads_started":1}
I20260806 08:59:25.819706 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.808869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.22.98.1:37927)
0806 08:59:25.809621 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.809632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.810219 (+ 587us) server_negotiation.cc:407] Connection header received
0806 08:59:25.810545 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.810556 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.810850 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.811058 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:59:25.812252 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.814054 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.816731 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.818037 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.818165 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.818402 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.818767 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.819003 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":357,"threads_started":1}
I20260806 08:59:25.820135 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.805782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.22.98.2:45405)
0806 08:59:25.811362 (+ 5580us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.811371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.811396 (+ 25us) server_negotiation.cc:407] Connection header received
0806 08:59:25.811471 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.811476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.811699 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.812618 (+ 919us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:59:25.812849 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.814336 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.816756 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.818037 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.818156 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.818372 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.819646 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.819814 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5383,"thread_start_us":5282,"threads_started":1}
I20260806 08:59:25.819468 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.808311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:37927 (local address 127.0.0.1:56028)
0806 08:59:25.810092 (+ 1781us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.810113 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.810338 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.811427 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.811436 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.811470 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:59:25.811992 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.812006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.814266 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.814272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.816573 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.816586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.818293 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.818300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.818520 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.818581 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.818702 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.818929 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"mutex_wait_us":139,"thread_start_us":1494,"threads_started":1}
I20260806 08:59:25.824054 6739 tablet_service.cc:1467] Processing CreateTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 (DEFAULT_TABLE table=test-table [id=7b5933d540ed480281f003255be205f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:59:25.824918 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.806480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:46039 (local address 127.0.0.1:41442)
0806 08:59:25.810724 (+ 4244us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.810742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.810862 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.815607 (+ 4745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.815612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.815638 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:59:25.816139 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.816152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.820061 (+ 3909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.820065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.822064 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.822079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.824121 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.824129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.824297 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.824333 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.824436 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.824574 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1736,"thread_start_us":373,"threads_started":1}
I20260806 08:59:25.825029 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.806537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.22.98.3:46039)
0806 08:59:25.814945 (+ 8408us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.814954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.814986 (+ 32us) server_negotiation.cc:407] Connection header received
0806 08:59:25.815085 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.815093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.815368 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.815598 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:59:25.816360 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.819869 (+ 3509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.822326 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.823887 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.824037 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.824288 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.824499 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.824672 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8230,"thread_start_us":8066,"threads_started":1}
I20260806 08:59:25.824407 6533 tablet_service.cc:1467] Processing CreateTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 (DEFAULT_TABLE table=test-table [id=7b5933d540ed480281f003255be205f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:59:25.826136 6739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c858fda03a44f880fd9ec5469a17f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:59:25.826566 6533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c858fda03a44f880fd9ec5469a17f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:59:25.829277 6943 tablet_service.cc:1467] Processing CreateTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 (DEFAULT_TABLE table=test-table [id=7b5933d540ed480281f003255be205f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 08:59:25.830868 6943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c858fda03a44f880fd9ec5469a17f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 08:59:25.838383 7058 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Bootstrap starting.
I20260806 08:59:25.842536 7058 tablet_bootstrap.cc:654] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Neither blocks nor log segments found. Creating new log.
I20260806 08:59:25.842625 7060 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Bootstrap starting.
I20260806 08:59:25.844061 7058 log.cc:824] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Log is configured to *not* fsync() on all Append() calls
I20260806 08:59:25.846463 7060 tablet_bootstrap.cc:654] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Neither blocks nor log segments found. Creating new log.
I20260806 08:59:25.847591 7058 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: No bootstrap required, opened a new log
I20260806 08:59:25.847637 7062 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Bootstrap starting.
I20260806 08:59:25.847812 7060 log.cc:824] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Log is configured to *not* fsync() on all Append() calls
I20260806 08:59:25.847898 7058 ts_tablet_manager.cc:1397] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 08:59:25.851431 7060 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: No bootstrap required, opened a new log
I20260806 08:59:25.851431 7062 tablet_bootstrap.cc:654] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Neither blocks nor log segments found. Creating new log.
I20260806 08:59:25.851776 7060 ts_tablet_manager.cc:1397] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 08:59:25.852713 7062 log.cc:824] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Log is configured to *not* fsync() on all Append() calls
I20260806 08:59:25.855937 7062 tablet_bootstrap.cc:492] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: No bootstrap required, opened a new log
I20260806 08:59:25.855867 7058 raft_consensus.cc:348] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.856216 7062 ts_tablet_manager.cc:1397] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260806 08:59:25.856249 7058 raft_consensus.cc:374] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:59:25.856346 7058 raft_consensus.cc:729] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e7f51db1944b1b8737c4da81ad8d98, State: Initialized, Role: FOLLOWER
I20260806 08:59:25.856937 7058 consensus_queue.cc:260] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.857982 6836 heartbeater.cc:503] Master 127.22.98.62:37091 was elected leader, sending a full tablet report...
I20260806 08:59:25.858675 7058 ts_tablet_manager.cc:1428] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260806 08:59:25.862026 7060 raft_consensus.cc:348] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.862486 7060 raft_consensus.cc:374] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:59:25.862591 7060 raft_consensus.cc:729] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9945a44777ca414aae1148f8fce586da, State: Initialized, Role: FOLLOWER
I20260806 08:59:25.863291 7060 consensus_queue.cc:260] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.865285 7060 ts_tablet_manager.cc:1428] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260806 08:59:25.866701 7062 raft_consensus.cc:348] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.867152 7062 raft_consensus.cc:374] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 08:59:25.867271 7062 raft_consensus.cc:729] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9729e47166c443fbbe20872ee77630c, State: Initialized, Role: FOLLOWER
I20260806 08:59:25.867946 7062 consensus_queue.cc:260] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.869056 7033 heartbeater.cc:503] Master 127.22.98.62:37091 was elected leader, sending a full tablet report...
I20260806 08:59:25.869828 7062 ts_tablet_manager.cc:1428] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260806 08:59:25.890352 6839 tablet.cc:2369] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 08:59:25.923023 6629 tablet.cc:2369] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:59:25.937537 7070 raft_consensus.cc:482] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 08:59:25.937846 7070 raft_consensus.cc:504] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.939587 7070 leader_election.cc:288] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e7f51db1944b1b8737c4da81ad8d98 (127.22.98.2:45405), 9945a44777ca414aae1148f8fce586da (127.22.98.1:37927)
I20260806 08:59:25.945298 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.939983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:45405 (local address 127.22.98.3:57865)
0806 08:59:25.940174 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.940196 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.940319 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.940689 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.940693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.940729 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:59:25.940969 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.940982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.942031 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.942039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.943590 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.943605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.944717 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.944722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.944885 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.944910 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.944980 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.945082 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 08:59:25.945519 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.939964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:57865 (local address 127.22.98.2:45405)
0806 08:59:25.940237 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.940242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.940262 (+ 20us) server_negotiation.cc:407] Connection header received
0806 08:59:25.940469 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.940474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.940552 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.940650 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:59:25.941137 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.941881 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.943772 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.944564 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.944671 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.944754 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.945029 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.945202 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 08:59:25.946465 6767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c858fda03a44f880fd9ec5469a17f1" candidate_uuid: "f9729e47166c443fbbe20872ee77630c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" is_pre_election: true
I20260806 08:59:25.946969 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.940539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:35019 (local address 127.22.98.1:37927)
0806 08:59:25.940781 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:25.940788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:25.941384 (+ 596us) server_negotiation.cc:407] Connection header received
0806 08:59:25.941534 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:25.941538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:25.941595 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:25.941669 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 08:59:25.942298 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.942994 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.944823 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.946047 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.946185 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.946293 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:25.946585 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 08:59:25.946727 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 08:59:25.947023 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:25.940382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:37927 (local address 127.22.98.3:35019)
0806 08:59:25.941282 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 08:59:25.941304 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 08:59:25.941415 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 08:59:25.941694 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 08:59:25.941698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 08:59:25.941722 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 08:59:25.942108 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.942124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.943153 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.943159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.944644 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 08:59:25.944661 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 08:59:25.946254 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 08:59:25.946260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 08:59:25.946420 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:25.946455 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 08:59:25.946546 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 08:59:25.946683 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":613,"threads_started":1}
I20260806 08:59:25.947008 6767 raft_consensus.cc:2454] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9729e47166c443fbbe20872ee77630c in term 0.
I20260806 08:59:25.947774 6874 leader_election.cc:302] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e7f51db1944b1b8737c4da81ad8d98, f9729e47166c443fbbe20872ee77630c; no voters:
I20260806 08:59:25.948259 7070 raft_consensus.cc:2781] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 08:59:25.948410 7070 raft_consensus.cc:482] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 08:59:25.948246 6563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c858fda03a44f880fd9ec5469a17f1" candidate_uuid: "f9729e47166c443fbbe20872ee77630c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945a44777ca414aae1148f8fce586da" is_pre_election: true
I20260806 08:59:25.948508 7070 raft_consensus.cc:3037] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:59:25.948748 6563 raft_consensus.cc:2454] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9729e47166c443fbbe20872ee77630c in term 0.
I20260806 08:59:25.950932 7070 raft_consensus.cc:504] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.951525 7070 leader_election.cc:288] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [CANDIDATE]: Term 1 election: Requested vote from peers b8e7f51db1944b1b8737c4da81ad8d98 (127.22.98.2:45405), 9945a44777ca414aae1148f8fce586da (127.22.98.1:37927)
I20260806 08:59:25.951988 6767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c858fda03a44f880fd9ec5469a17f1" candidate_uuid: "f9729e47166c443fbbe20872ee77630c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e7f51db1944b1b8737c4da81ad8d98"
I20260806 08:59:25.952142 6563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c858fda03a44f880fd9ec5469a17f1" candidate_uuid: "f9729e47166c443fbbe20872ee77630c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9945a44777ca414aae1148f8fce586da"
I20260806 08:59:25.952181 6767 raft_consensus.cc:3037] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:59:25.952497 6563 raft_consensus.cc:3037] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 0 FOLLOWER]: Advancing to term 1
I20260806 08:59:25.954679 6767 raft_consensus.cc:2454] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9729e47166c443fbbe20872ee77630c in term 1.
I20260806 08:59:25.954787 6563 raft_consensus.cc:2454] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9729e47166c443fbbe20872ee77630c in term 1.
I20260806 08:59:25.955134 6874 leader_election.cc:302] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e7f51db1944b1b8737c4da81ad8d98, f9729e47166c443fbbe20872ee77630c; no voters:
I20260806 08:59:25.955425 7070 raft_consensus.cc:2781] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 08:59:25.955933 7070 raft_consensus.cc:686] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 1 LEADER]: Becoming Leader. State: Replica: f9729e47166c443fbbe20872ee77630c, State: Running, Role: LEADER
I20260806 08:59:25.956492 7070 consensus_queue.cc:237] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } }
I20260806 08:59:25.961369 6324 catalog_manager.cc:5495] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c reported cstate change: term changed from 0 to 1, leader changed from <none> to f9729e47166c443fbbe20872ee77630c (127.22.98.3). New cstate: current_term: 1 leader_uuid: "f9729e47166c443fbbe20872ee77630c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9729e47166c443fbbe20872ee77630c" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46039 } health_report { overall_health: HEALTHY } } }
W20260806 08:59:25.983379 7035 tablet.cc:2369] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 08:59:26.014816 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:26.009410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.22.98.3:46039)
0806 08:59:26.009593 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:26.009598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:26.009616 (+ 18us) server_negotiation.cc:407] Connection header received
0806 08:59:26.009742 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:26.009745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:26.009826 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:26.009956 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 08:59:26.010329 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.011106 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.012547 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.012964 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.013103 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:26.013728 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 08:59:26.014274 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 08:59:26.014406 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:26.014535 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 08:59:26.014606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 08:59:26.024374 6767 raft_consensus.cc:1261] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 1 FOLLOWER]: Refusing update from remote peer f9729e47166c443fbbe20872ee77630c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:59:26.024374 6563 raft_consensus.cc:1261] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 1 FOLLOWER]: Refusing update from remote peer f9729e47166c443fbbe20872ee77630c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 08:59:26.025584 7073 consensus_queue.cc:1035] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e7f51db1944b1b8737c4da81ad8d98" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:59:26.026118 7070 consensus_queue.cc:1035] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9945a44777ca414aae1148f8fce586da" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 08:59:26.055444 6323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:59:26.055819 6323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 08:59:26.059769 6323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfe5c112b3c444b6961ee9112cd4f42f: Sending DeleteTablet for 3 replicas of tablet 87c858fda03a44f880fd9ec5469a17f1
I20260806 08:59:26.061005 6943 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:59:26 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:41442
I20260806 08:59:26.060981 6739 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:59:26 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:42720
I20260806 08:59:26.061709 6533 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c858fda03a44f880fd9ec5469a17f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 08:59:26 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:56028
I20260806 08:59:26.062074 7095 tablet_replica.cc:330] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: stopping tablet replica
I20260806 08:59:26.062707 7095 raft_consensus.cc:2229] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:59:26.062872 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 6430
I20260806 08:59:26.063655 7095 raft_consensus.cc:2258] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:59:26.064467 7096 tablet_replica.cc:330] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: stopping tablet replica
I20260806 08:59:26.065385 7096 raft_consensus.cc:2229] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 1 LEADER]: Raft consensus shutting down.
I20260806 08:59:26.066340 7096 raft_consensus.cc:2258] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:59:26.068831 7098 tablet_replica.cc:330] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: stopping tablet replica
I20260806 08:59:26.069020 7095 ts_tablet_manager.cc:1905] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:59:26.069473 7098 raft_consensus.cc:2229] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 08:59:26.070152 7098 raft_consensus.cc:2258] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 08:59:26.072593 7098 ts_tablet_manager.cc:1905] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 08:59:26.073127 7096 ts_tablet_manager.cc:1905] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 08:59:26 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 08:59:26.080695 7098 ts_tablet_manager.cc:1918] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:59:26.080907 7098 log.cc:1196] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/87c858fda03a44f880fd9ec5469a17f1
I20260806 08:59:26.081492 7098 ts_tablet_manager.cc:1939] T 87c858fda03a44f880fd9ec5469a17f1 P 9945a44777ca414aae1148f8fce586da: Deleting consensus metadata
I20260806 08:59:26.083822 6306 catalog_manager.cc:4841] TS 9945a44777ca414aae1148f8fce586da (127.22.98.1:37927): tablet 87c858fda03a44f880fd9ec5469a17f1 (table test-table [id=7b5933d540ed480281f003255be205f1]) successfully deleted
I20260806 08:59:26.085367 7095 ts_tablet_manager.cc:1918] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:59:26.085613 7095 log.cc:1196] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/87c858fda03a44f880fd9ec5469a17f1
I20260806 08:59:26.086220 7095 ts_tablet_manager.cc:1939] T 87c858fda03a44f880fd9ec5469a17f1 P b8e7f51db1944b1b8737c4da81ad8d98: Deleting consensus metadata
I20260806 08:59:26.088630 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:26.064146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.22.98.1:37927)
0806 08:59:26.064330 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:26.064336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:26.068115 (+ 3779us) server_negotiation.cc:407] Connection header received
0806 08:59:26.068340 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:26.068346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:26.068445 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:26.068570 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:26.071154 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.072267 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.074337 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.074999 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.075120 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:26.081386 (+ 6266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:26.081401 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:26.081421 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:26.081485 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:26.085362 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.086314 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.086325 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.086329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.086487 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.086907 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.086913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.086916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.087298 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:26.087624 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:26.087981 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 08:59:26.088250 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 08:59:26.089375 6619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:59:26.089766 7096 ts_tablet_manager.cc:1918] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 08:59:26.089958 7096 log.cc:1196] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/87c858fda03a44f880fd9ec5469a17f1
I20260806 08:59:26.091321 6306 catalog_manager.cc:4841] TS b8e7f51db1944b1b8737c4da81ad8d98 (127.22.98.2:45405): tablet 87c858fda03a44f880fd9ec5469a17f1 (table test-table [id=7b5933d540ed480281f003255be205f1]) successfully deleted
I20260806 08:59:26.091850 7096 ts_tablet_manager.cc:1939] T 87c858fda03a44f880fd9ec5469a17f1 P f9729e47166c443fbbe20872ee77630c: Deleting consensus metadata
I20260806 08:59:26.093983 6302 catalog_manager.cc:4841] TS f9729e47166c443fbbe20872ee77630c (127.22.98.3:46039): tablet 87c858fda03a44f880fd9ec5469a17f1 (table test-table [id=7b5933d540ed480281f003255be205f1]) successfully deleted
I20260806 08:59:26.242863 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 6430
I20260806 08:59:26.268194 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 6637
Aug 06 08:59:26 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 08:59:26.286484 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:26.269255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42730 (local address 127.22.98.2:45405)
0806 08:59:26.269424 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:26.269429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:26.269448 (+ 19us) server_negotiation.cc:407] Connection header received
0806 08:59:26.269509 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:26.269513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:26.269601 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:26.269731 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:26.271482 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.272372 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.274304 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.274772 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.274880 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:26.280465 (+ 5585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:26.280483 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:26.280501 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:26.280544 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:26.283475 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.284398 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.284407 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.284409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.284565 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.285003 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.285007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.285009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.285460 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:26.285732 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:26.286006 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 08:59:26.286207 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 08:59:26.287248 6827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:59:26.386293 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 6637
I20260806 08:59:26.415751 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 6845
Aug 06 08:59:26 dist-test-slave-jpj1 krb5kdc[5669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 08:59:26.435694 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 08:59:26.416723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.22.98.3:46039)
0806 08:59:26.416902 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 08:59:26.416907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 08:59:26.416928 (+ 21us) server_negotiation.cc:407] Connection header received
0806 08:59:26.417193 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 08:59:26.417198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 08:59:26.417266 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 08:59:26.417355 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 08:59:26.419263 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.420276 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.422222 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 08:59:26.422907 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 08:59:26.423022 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 08:59:26.428999 (+ 5977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 08:59:26.429022 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 08:59:26.429045 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 08:59:26.429101 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 08:59:26.432504 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.433321 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.433330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.433336 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.433501 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 08:59:26.433913 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 08:59:26.433917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 08:59:26.433919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 08:59:26.434353 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 08:59:26.434652 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 08:59:26.435004 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 08:59:26.435256 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 08:59:26.436446 7028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:59:26.559962 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 6845
I20260806 08:59:26.585433 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 6273
I20260806 08:59:26.586670 6365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 08:59:26.699429 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 6273
I20260806 08:59:26.728099 22920 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 08:59:27.074120 22920 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 08:59:27.074410 22920 mini_ranger.cc:66] Stopping Ranger...
I20260806 08:59:27.420569 22920 mini_ranger.cc:68] Stopped Ranger
2026-08-06 08:59:27.420 UTC [5715] LOG: received smart shutdown request
2026-08-06 08:59:27.425 UTC [5715] LOG: background worker "logical replication launcher" (PID 5727) exited with exit code 1
2026-08-06 08:59:27.425 UTC [5721] LOG: shutting down
2026-08-06 08:59:27.621 UTC [5715] LOG: database system is shut down
2026-08-06T08:59:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51230 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 08:59:27 dist-test-slave-jpj1 krb5kdc[7120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 08:59:27 dist-test-slave-jpj1 krb5kdc[7120](info): set up 2 sockets
Aug 06 08:59:27 dist-test-slave-jpj1 krb5kdc[7120](info): commencing operation
krb5kdc: starting...
W20260806 08:59:28.932201 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 08:59:29 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T08:59:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T08:59:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangeradmin_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangeradmin_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangerlookup_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangerlookup_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
I20260806 08:59:29.339807 22920 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6BBFDc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/postgres -l logfile start
2026-08-06 08:59:31.774 UTC [7161] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 08:59:31.774 UTC [7161] LOG: listening on IPv4 address "127.22.98.20", port 35031
2026-08-06 08:59:31.778 UTC [7161] LOG: listening on Unix socket "/tmp/.s.PGSQL.35031"
2026-08-06 08:59:31.799 UTC [7165] LOG: database system was shut down at 2026-08-06 08:59:30 UTC
2026-08-06 08:59:31.802 UTC [7161] LOG: database system is ready to accept connections
I20260806 08:59:32.860461 22920 mini_postgres.cc:96] Postgres bound to 35031
2026-08-06 08:59:32.883 UTC [7176] FATAL: database "slave" does not exist
127.22.98.20:35031 - accepting connections
I20260806 08:59:32.884248 22920 mini_ranger.cc:161] Starting Ranger...
I20260806 08:59:32.916283 22920 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 08:59:33.399909 22920 mini_ranger.cc:87] Created ranger Postgres database
I20260806 08:59:33.400038 22920 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin
2026-08-06 08:59:33,694 [I] DB FLAVOR :POSTGRES
2026-08-06 08:59:33,695 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 08:59:33,695 [I] Checking connection..
2026-08-06 08:59:33,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 08:59:33,966 [I] Checking connection passed.
2026-08-06 08:59:33,967 [I] --------- Verifying version history table ---------
2026-08-06 08:59:33,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:59:34,216 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 08:59:34,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:59:34,486 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 08:59:34,486 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 08:59:34,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 08:59:34.732 UTC [7269] WARNING: there is no transaction in progress
2026-08-06 08:59:34,739 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 08:59:34,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 08:59:35,011 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 08:59:35,011 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 08:59:35,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 08:59:35,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 08:59:35,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jpj1.c.gcp-upstream.internal','N') ;"
2026-08-06 08:59:35,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 08:59:36,078 [I] Table x_portal_user does not exist in database ranger
2026-08-06 08:59:36,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 08:59:36,326 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 08:59:36,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 08:59:36,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 08:59:36,834 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 08:59:36,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 08:59:37.872 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:37.888 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:37.904 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:37.920 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:37.935 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:37.953 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.009 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.016 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.030 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.041 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.051 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.065 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.077 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.084 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.091 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38.099 UTC [7422] WARNING: there is no transaction in progress
2026-08-06 08:59:38,731 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 08:59:38,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 08:59:38,994 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 08:59:38,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 08:59:39,259 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 08:59:39,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 08:59:39,525 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 08:59:39,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 08:59:39,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 08:59:40,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jpj1.c.gcp-upstream.internal';"
2026-08-06 08:59:40,308 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 08:59:40,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 08:59:40,553 [I] DB_PATCHES have already been applied
I20260806 08:59:40.560077 22920 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6BBFDc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 08:59:40.560175 22920 mini_ranger.cc:193] Using host: 127.22.98.20
I20260806 08:59:40.579936 22920 mini_ranger.cc:227] Ranger admin URL: http://127.22.98.20:39785
Aug 06, 2026 8:59:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:59:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 8:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 8:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 8:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 8:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 08:59:41.795472 22920 mini_ranger.cc:160] Time spent starting Ranger: real 8.911s user 0.008s sys 0.053s
Aug 06, 2026 9:00:01 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:00:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 9:00:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:00:01 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:00:02 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 09:00:03.379679 22920 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.98.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.98.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangerkms_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.98.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/rangerkms_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.98.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.98.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.98.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/HTTP_127.22.98.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 09:00:03 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006803, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 09:00:03.609817 22920 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 09:00:03.637487 22920 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 09:00:04.653244 22920 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 09:00:04.653348 22920 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms
I20260806 09:00:04.653390 22920 mini_ranger_kms.cc:227] Using postgres at 127.22.98.20:35031
2026-08-06 09:00:05,024 [I] DB FLAVOR :POSTGRES
2026-08-06 09:00:05,024 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 09:00:05,024 [I] Checking connection
2026-08-06 09:00:05,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 09:00:05,299 [I] connection success
2026-08-06 09:00:05,299 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 09:00:05,300 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 09:00:05,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 09:00:05,552 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 09:00:05,552 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 09:00:05,553 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 09:00:05,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.98.20:35031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 09:00:05,824 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 09:00:06.345242 22920 mini_ranger_kms.cc:310] Created kms service
I20260806 09:00:06.536983 22920 mini_ranger_kms.cc:326] Created kudu user
I20260806 09:00:06.584741 22920 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 09:00:06 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006803, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:07.014482 22920 mini_ranger_kms.cc:384] Added ranger policy
I20260806 09:00:07.014837 22920 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms:/tmp/dist-test-task6BBFDc/build/asan/bin/postgresql.jar:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6BBFDc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 09:00:07.014892 22920 mini_ranger_kms.cc:240] Using host: 127.22.98.20
I20260806 09:00:07.038720 22920 mini_ranger_kms.cc:276] Ranger KMS PID: 7681
I20260806 09:00:07.038882 22920 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.98.20:49889
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:00:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 09:00:09 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786006809, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:09.553136 22920 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.943s user 0.078s sys 0.334s
I20260806 09:00:09.553400 22920 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 09:00:09.553555 22920 mini_ranger_kms.cc:401] 127.22.98.20:49889/kms/v1/keys
Aug 06 09:00:09 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006809, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
Aug 06 09:00:10 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006809, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.98.20@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
Aug 06 09:00:12 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:00:12.692129 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34809
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34809
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:13.047240 7752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:13.047616 7752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:13.047750 7752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:13.055693 7752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:13.055830 7752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:13.055886 7752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:13.055970 7752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:13.056035 7752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:13.067763 7752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.98.20:49889/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34809
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:13.069406 7752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:13.070925 7752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:13.083582 7757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:13.083503 7758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:13.083578 7761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:13.084414 7752 server_base.cc:1034] running on GCE node
I20260806 09:00:13.085261 7752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:13.086774 7752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:13.088006 7752 hybrid_clock.cc:648] HybridClock initialized: now 1786006813087957 us; error 33 us; skew 500 ppm
I20260806 09:00:13.088408 7752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:13 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006813, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:13.091837 7752 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:13.093539 7752 webserver.cc:460] Webserver started at http://127.22.98.62:39367/ using document root <none> and password file <none>
I20260806 09:00:13.094260 7752 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:13.094377 7752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:13.094724 7752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:00:13 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006813, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:13.146047 7752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "608a1a3be79442cf89dd5a9919bbdf26"
format_stamp: "Formatted at 2026-08-06 09:00:13 on dist-test-slave-jpj1"
server_key: "6a7c3b3aed5adc4e10aca61cd753417a"
server_key_iv: "f5ccbf4cf4ffad19aa4c5933e7fcd088"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:13.146967 7752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "608a1a3be79442cf89dd5a9919bbdf26"
format_stamp: "Formatted at 2026-08-06 09:00:13 on dist-test-slave-jpj1"
server_key: "6a7c3b3aed5adc4e10aca61cd753417a"
server_key_iv: "f5ccbf4cf4ffad19aa4c5933e7fcd088"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:13.153158 7752 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 09:00:13.170970 7771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:13.172606 7752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:00:13.172799 7752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "608a1a3be79442cf89dd5a9919bbdf26"
format_stamp: "Formatted at 2026-08-06 09:00:13 on dist-test-slave-jpj1"
server_key: "6a7c3b3aed5adc4e10aca61cd753417a"
server_key_iv: "f5ccbf4cf4ffad19aa4c5933e7fcd088"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:13.173118 7752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:13.196844 7752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:13.201061 7752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:13.201385 7752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:13.220013 7752 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34809
I20260806 09:00:13.220048 7850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34809 every 8 connection(s)
I20260806 09:00:13.221588 7752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:13.225203 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 7752
I20260806 09:00:13.225412 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:00:13.225831 7853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:13.226094 22920 external_mini_cluster.cc:1428] Setting key 40561110c770f6643a868c36fd796b50
I20260806 09:00:13.241026 7853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Bootstrap starting.
Aug 06 09:00:13 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:13.244462 7853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:13.245573 7853 log.cc:824] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:13.248817 7853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: No bootstrap required, opened a new log
I20260806 09:00:13.249706 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:13.228203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.22.98.62:34809)
0806 09:00:13.229132 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:13.229149 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:13.229199 (+ 50us) server_negotiation.cc:407] Connection header received
0806 09:00:13.230786 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:13.230811 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:13.231225 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:13.231645 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:13.233501 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:13.234842 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:13.236740 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:13.237694 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:13.237877 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:13.242601 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:13.242614 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:13.242630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:13.242675 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:13.245640 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:13.246337 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:13.246344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:13.246351 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:13.246527 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:13.246914 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:13.246918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:13.246921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:13.247434 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:13.247770 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:13.248203 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 09:00:13.248617 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":456,"threads_started":1}
I20260806 09:00:13.257282 7853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:13.257743 7853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:13.257870 7853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608a1a3be79442cf89dd5a9919bbdf26, State: Initialized, Role: FOLLOWER
I20260806 09:00:13.258648 7853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:13.258939 7853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:13.259056 7853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:13.259207 7853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:13.261541 7853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:13.262092 7853 leader_election.cc:302] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 608a1a3be79442cf89dd5a9919bbdf26; no voters:
I20260806 09:00:13.262683 7853 leader_election.cc:288] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:00:13.262830 7860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:13.263502 7860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 LEADER]: Becoming Leader. State: Replica: 608a1a3be79442cf89dd5a9919bbdf26, State: Running, Role: LEADER
I20260806 09:00:13.264084 7860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:13.264878 7853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:13.266724 7863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608a1a3be79442cf89dd5a9919bbdf26. Latest consensus state: current_term: 1 leader_uuid: "608a1a3be79442cf89dd5a9919bbdf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } } }
I20260806 09:00:13.267100 7863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:13.267735 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "608a1a3be79442cf89dd5a9919bbdf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } } }
I20260806 09:00:13.268051 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:13.269595 7868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:13.275528 7868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:13.276332 7853 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 09:00:13.295818 7868 catalog_manager.cc:1347] Generated new cluster ID: a0b9864d014c462fa276f9c562ce4dc8
I20260806 09:00:13.295977 7868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:13.308019 7868 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:00:13.309629 7868 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:13.316056 7868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Generated new TSK 0
I20260806 09:00:13.316967 7868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:00:14.143886 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:14 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 09:00:14.642874 7912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:14.643297 7912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:14.643527 7912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:14.654156 7912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:14.654297 7912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:14.654481 7912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:14.666097 7912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:14.667912 7912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:14.669548 7912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:14.686131 7933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:14.686116 7934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:14.686529 7936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:14.688439 7912 server_base.cc:1034] running on GCE node
I20260806 09:00:14.689149 7912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:14.690234 7912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:14.691473 7912 hybrid_clock.cc:648] HybridClock initialized: now 1786006814691416 us; error 47 us; skew 500 ppm
I20260806 09:00:14.691877 7912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:14 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:14.695456 7912 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:14.697194 7912 webserver.cc:460] Webserver started at http://127.22.98.1:35901/ using document root <none> and password file <none>
I20260806 09:00:14.697988 7912 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:14.698099 7912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:14.698426 7912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:00:14 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:14.729446 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "e721bcad07394d128942eabbb9a9b042"
format_stamp: "Formatted at 2026-08-06 09:00:14 on dist-test-slave-jpj1"
server_key: "fd7125e6f3e1742a4339f4456829ad52"
server_key_iv: "d778b3ec07ccf2a1a6f2cf565caa8e23"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:14.730803 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "e721bcad07394d128942eabbb9a9b042"
format_stamp: "Formatted at 2026-08-06 09:00:14 on dist-test-slave-jpj1"
server_key: "fd7125e6f3e1742a4339f4456829ad52"
server_key_iv: "d778b3ec07ccf2a1a6f2cf565caa8e23"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:14.743667 7912 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.013s sys 0.000s
I20260806 09:00:14.773774 7943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:14.776028 7912 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.004s
I20260806 09:00:14.776264 7912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "e721bcad07394d128942eabbb9a9b042"
format_stamp: "Formatted at 2026-08-06 09:00:14 on dist-test-slave-jpj1"
server_key: "fd7125e6f3e1742a4339f4456829ad52"
server_key_iv: "d778b3ec07ccf2a1a6f2cf565caa8e23"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:14.776561 7912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:14.816354 7912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:14.822028 7912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:14.822510 7912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:14.823899 7912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:14.826000 7912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:14.826135 7912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 09:00:14.826306 7912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:14.826408 7912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:00:14 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:15.060076 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:43403 every 8 connection(s)
I20260806 09:00:15.060515 7912 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:43403
I20260806 09:00:15.062505 7912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:15.064455 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 7912
I20260806 09:00:15.064702 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:00:15.065555 22920 external_mini_cluster.cc:1428] Setting key d75b0fccd9cb5e006913de6f42038778
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Aug 06 09:00:15 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
I20260806 09:00:15.140115 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:15.071846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:37935 (local address 127.22.98.62:34809)
0806 09:00:15.091344 (+ 19498us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:15.091351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:15.091376 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:00:15.091448 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:15.091453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:15.091588 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:15.091738 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:15.098919 (+ 7181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.100139 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.102905 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.103599 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.103736 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:15.131592 (+ 27856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:15.131602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:15.131606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:15.131667 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:15.135702 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:15.136974 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:15.136981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:15.136985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:15.137167 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:15.137783 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:15.137788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:15.137792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:15.138181 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:15.138319 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:15.139405 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0806 09:00:15.139718 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19346,"thread_start_us":20005,"threads_started":1}
I20260806 09:00:15.142904 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:15.072487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.1:37935)
0806 09:00:15.087121 (+ 14634us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:15.087203 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:15.089467 (+ 2264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:15.092361 (+ 2894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:15.092377 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:15.097602 (+ 5225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:15.098587 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:15.098634 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.100411 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.100417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:15.102625 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:15.102662 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.114732 (+ 12070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.114744 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:15.115186 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:15.116275 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:15.116317 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:15.131201 (+ 14884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:15.136027 (+ 4826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:15.136040 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:15.136066 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:15.136794 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:15.137312 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:15.137318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:15.137322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:15.137629 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:15.138410 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:15.138419 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:15.138940 (+ 521us) client_negotiation.cc:769] Sending connection context
0806 09:00:15.140583 (+ 1643us) client_negotiation.cc:240] Negotiation successful
0806 09:00:15.141303 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14276,"thread_start_us":17382,"threads_started":1}
I20260806 09:00:15.145385 8103 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:15.145743 8103 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:15.146684 8103 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:15.149473 7797 ts_manager.cc:194] Registered new tserver with Master: e721bcad07394d128942eabbb9a9b042 (127.22.98.1:43403)
I20260806 09:00:15.151633 7797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:37935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
W20260806 09:00:15.178766 22920 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.98.2: real 0.110s user 0.000s sys 0.025s
I20260806 09:00:15.179708 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:15 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006814, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
W20260806 09:00:15.717849 8109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:15.718340 8109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:15.718529 8109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:15.729173 8109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:15.729380 8109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:15.729640 8109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:15.745450 8109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:15.747527 8109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:15.749449 8109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:15.767375 8131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:15.771965 8130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:15.772907 8136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:15.773772 8109 server_base.cc:1034] running on GCE node
I20260806 09:00:15.774636 8109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:15.775918 8109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:15.777225 8109 hybrid_clock.cc:648] HybridClock initialized: now 1786006815777149 us; error 60 us; skew 500 ppm
I20260806 09:00:15.777829 8109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:15 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006815, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:15.782588 8109 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:15.785071 8109 webserver.cc:460] Webserver started at http://127.22.98.2:44623/ using document root <none> and password file <none>
I20260806 09:00:15.786099 8109 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:15.786316 8109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:15.786728 8109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:00:15 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006815, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:15.812512 8109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "ea341218b50d49a2b38cd04139230bb2"
format_stamp: "Formatted at 2026-08-06 09:00:15 on dist-test-slave-jpj1"
server_key: "b3af806ef6dea29e66833fa5f58ed3db"
server_key_iv: "c42c44fe3ca2aff5e971e85c79350d13"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:15.814095 8109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "ea341218b50d49a2b38cd04139230bb2"
format_stamp: "Formatted at 2026-08-06 09:00:15 on dist-test-slave-jpj1"
server_key: "b3af806ef6dea29e66833fa5f58ed3db"
server_key_iv: "c42c44fe3ca2aff5e971e85c79350d13"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:15.828272 8109 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.010s sys 0.002s
I20260806 09:00:15.848821 8146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:15.850986 8109 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 09:00:15.851284 8109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "ea341218b50d49a2b38cd04139230bb2"
format_stamp: "Formatted at 2026-08-06 09:00:15 on dist-test-slave-jpj1"
server_key: "b3af806ef6dea29e66833fa5f58ed3db"
server_key_iv: "c42c44fe3ca2aff5e971e85c79350d13"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:15.851670 8109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:15.888197 8109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:15.893712 8109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:15.894064 8109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:15.895341 8109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:15.897267 8109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:15.897367 8109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:15.897552 8109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:15.897622 8109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:15.936699 8109 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:34207
I20260806 09:00:15.936744 8311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:34207 every 8 connection(s)
I20260806 09:00:15.938482 8109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:15.948012 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 8109
I20260806 09:00:15.948199 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:00:15.948721 22920 external_mini_cluster.cc:1428] Setting key 9985aa44dcf488b44ca9158fdfa4f9f1
Aug 06 09:00:15 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006815, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
I20260806 09:00:15.991925 8315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:15.942935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:52471 (local address 127.22.98.62:34809)
0806 09:00:15.966013 (+ 23078us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:15.966020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:15.967604 (+ 1584us) server_negotiation.cc:407] Connection header received
0806 09:00:15.970770 (+ 3166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:15.970777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:15.970894 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:15.971026 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:15.974137 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.975277 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.978227 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.978856 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.978977 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:15.984232 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:15.984240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:15.984245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:15.984293 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:15.987824 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:15.988890 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:15.988897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:15.988900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:15.989066 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:15.989678 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:15.989683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:15.989686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:15.990066 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:15.990199 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:15.991271 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0806 09:00:15.991512 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22760,"thread_start_us":22950,"threads_started":1}
I20260806 09:00:15.994494 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:15.942273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.2:52471)
0806 09:00:15.966662 (+ 24389us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:15.966733 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:15.969191 (+ 2458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:15.971960 (+ 2769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:15.971977 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:15.972800 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:15.973867 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:15.973899 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.975494 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.975500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:15.977978 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:15.978006 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:15.979350 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:15.979358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:15.979781 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:15.980620 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:15.980653 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:15.983791 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:15.988021 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:15.988032 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:15.988055 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:15.988706 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:15.989216 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:15.989220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:15.989224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:15.989508 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:15.990307 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:15.990317 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:15.990795 (+ 478us) client_negotiation.cc:769] Sending connection context
0806 09:00:15.992481 (+ 1686us) client_negotiation.cc:240] Negotiation successful
0806 09:00:15.993077 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24010,"thread_start_us":27185,"threads_started":1}
I20260806 09:00:15.996416 8312 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:15.996729 8312 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:15.997581 8312 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:15.999393 7797 ts_manager.cc:194] Registered new tserver with Master: ea341218b50d49a2b38cd04139230bb2 (127.22.98.2:34207)
I20260806 09:00:16.000504 7797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:52471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:00:16.027930 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:16.155418 8103 heartbeater.cc:503] Master 127.22.98.62:34809 was elected leader, sending a full tablet report...
W20260806 09:00:16.463990 8319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:16.464329 8319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:16.464480 8319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:16.472483 8319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:16.472625 8319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:16.472777 8319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:00:16.483248 8319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:16.485122 8319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:16.486444 8319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:16.501641 8328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:16.502353 8326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:16.502378 8325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:16.503688 8319 server_base.cc:1034] running on GCE node
I20260806 09:00:16.504482 8319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:16.505606 8319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:16.506876 8319 hybrid_clock.cc:648] HybridClock initialized: now 1786006816506797 us; error 67 us; skew 500 ppm
I20260806 09:00:16.507280 8319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006816, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:16.510951 8319 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:16.512691 8319 webserver.cc:460] Webserver started at http://127.22.98.3:34181/ using document root <none> and password file <none>
I20260806 09:00:16.513595 8319 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:16.513773 8319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:16.514112 8319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006816, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:16.533607 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "b5e5491f93ed457eaaa56eacade8999a"
format_stamp: "Formatted at 2026-08-06 09:00:16 on dist-test-slave-jpj1"
server_key: "5a263999cf9dab3413c295124a0c9444"
server_key_iv: "edba376a0795a5f8d81820d6fc0ba25e"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:16.535017 8319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "b5e5491f93ed457eaaa56eacade8999a"
format_stamp: "Formatted at 2026-08-06 09:00:16 on dist-test-slave-jpj1"
server_key: "5a263999cf9dab3413c295124a0c9444"
server_key_iv: "edba376a0795a5f8d81820d6fc0ba25e"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:16.541947 8319 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 09:00:16.561964 8336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:16.564078 8319 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 09:00:16.564392 8319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "b5e5491f93ed457eaaa56eacade8999a"
format_stamp: "Formatted at 2026-08-06 09:00:16 on dist-test-slave-jpj1"
server_key: "5a263999cf9dab3413c295124a0c9444"
server_key_iv: "edba376a0795a5f8d81820d6fc0ba25e"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:16.564761 8319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:16.580981 8319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:16.585553 8319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:16.585880 8319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:16.586923 8319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:16.588562 8319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:16.588646 8319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:16.588774 8319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:16.588837 8319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:16.622957 8319 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:46357
I20260806 09:00:16.622975 8494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:46357 every 8 connection(s)
I20260806 09:00:16.624541 8319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:16.626178 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 8319
I20260806 09:00:16.626362 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:00:16.626929 22920 external_mini_cluster.cc:1428] Setting key 700c13b3e5b7811e39e8bf386026be6e
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006816, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:16.648331 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:16.627634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:32911 (local address 127.22.98.62:34809)
0806 09:00:16.628452 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:16.628458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:16.631596 (+ 3138us) server_negotiation.cc:407] Connection header received
0806 09:00:16.631802 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:16.631806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:16.631884 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:16.631972 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:16.633773 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.634467 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.636955 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.637430 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.637559 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:16.641805 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:16.641812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:16.641817 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:16.641858 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:16.644950 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.645796 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.645802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.645806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.645949 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.646445 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.646450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.646453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.646759 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:16.646901 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:16.647696 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 09:00:16.647945 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":423,"threads_started":1}
I20260806 09:00:16.649305 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:16.628235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.3:32911)
0806 09:00:16.629157 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:16.629209 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:16.631487 (+ 2278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:16.632257 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:16.632269 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:16.632824 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:16.633615 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:16.633638 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.634605 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.634610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:16.636629 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:16.636660 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.637718 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.637726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:16.638052 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:16.638829 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:16.638863 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:16.641504 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:16.645125 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:16.645136 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:16.645154 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:16.645648 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:16.646071 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:16.646077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:16.646080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:16.646311 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:16.646895 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:16.646903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:16.647264 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 09:00:16.647518 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 09:00:16.648126 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":404,"threads_started":1}
I20260806 09:00:16.651017 8495 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:16.651352 8495 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:16.652127 8495 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:16.653883 7797 ts_manager.cc:194] Registered new tserver with Master: b5e5491f93ed457eaaa56eacade8999a (127.22.98.3:46357)
I20260806 09:00:16.654948 7797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:32911
I20260806 09:00:16.656776 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:00:16.659052 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 7912
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:00:16.679198 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:16.660335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.22.98.1:43403)
0806 09:00:16.661022 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:16.661031 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:16.661061 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:00:16.661134 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:16.661141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:16.661365 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:16.661503 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:16.663415 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.664596 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.666799 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.667443 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.667615 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:16.672831 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:16.672849 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:16.672867 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:16.672917 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:16.676006 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.676822 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.676831 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.676833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.676999 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.677423 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.677427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.677429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.677821 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:16.678153 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:16.678504 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 09:00:16.678828 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":335,"threads_started":1}
I20260806 09:00:16.680543 8097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:16.794770 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 7912
I20260806 09:00:16.815332 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 8109
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:00:16.839963 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:16.820920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.22.98.2:34207)
0806 09:00:16.821470 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:16.821479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:16.821506 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:00:16.821574 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:16.821581 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:16.821832 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:16.821981 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:16.823704 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.824802 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.826813 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.827443 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.827597 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:16.832738 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:16.832759 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:16.832785 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:16.832851 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:16.836512 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.837379 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.837390 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.837394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.837585 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.838157 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.838162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.838164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.838587 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:16.838903 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:16.839280 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 09:00:16.839604 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":283,"threads_started":1}
I20260806 09:00:16.841363 8303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:16.925309 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 8109
I20260806 09:00:16.958297 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 8319
Aug 06 09:00:16 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:00:16.980171 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:16.959522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.22.98.3:46357)
0806 09:00:16.960276 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:16.960284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:16.960316 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:00:16.960381 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:16.960388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:16.960615 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:16.960791 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:16.962517 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.963668 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.965790 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:16.966404 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:16.966539 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:16.972677 (+ 6138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:16.972692 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:16.972710 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:16.972757 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:16.976587 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.977521 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.977532 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.977535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.977733 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:16.978184 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:16.978197 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:16.978201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:16.978714 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:16.979072 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:16.979398 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 09:00:16.979757 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":398,"threads_started":1}
I20260806 09:00:16.981714 8487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:17.069478 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 8319
I20260806 09:00:17.094452 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 7752
I20260806 09:00:17.095952 7844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:17.205523 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 7752
I20260806 09:00:17.237202 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34809
--webserver_interface=127.22.98.62
--webserver_port=39367
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34809
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:17.598296 8511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:17.598623 8511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:17.598749 8511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:17.606199 8511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:17.606317 8511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:17.606375 8511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:17.606459 8511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:17.606525 8511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:17.617084 8511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34809
--ranger_config_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=39367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:17.618454 8511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:17.619884 8511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:17.632071 8523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:17.632140 8517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:17.632112 8519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:17.632855 8511 server_base.cc:1034] running on GCE node
I20260806 09:00:17.633678 8511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:17.635174 8511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:17.636391 8511 hybrid_clock.cc:648] HybridClock initialized: now 1786006817636351 us; error 48 us; skew 500 ppm
I20260806 09:00:17.636768 8511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:17 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006817, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:17.639976 8511 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:17.641609 8511 webserver.cc:460] Webserver started at http://127.22.98.62:39367/ using document root <none> and password file <none>
I20260806 09:00:17.642293 8511 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:17.642411 8511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:00:17 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006817, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:17.722995 8511 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 09:00:17.726476 8533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:17.728206 8511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:00:17.728439 8511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "608a1a3be79442cf89dd5a9919bbdf26"
format_stamp: "Formatted at 2026-08-06 09:00:13 on dist-test-slave-jpj1"
server_key: "6a7c3b3aed5adc4e10aca61cd753417a"
server_key_iv: "f5ccbf4cf4ffad19aa4c5933e7fcd088"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:17.729447 8511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:17.746868 8511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:17.751245 8511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:17.751593 8511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:17.770468 8511 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34809
I20260806 09:00:17.770489 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34809 every 8 connection(s)
I20260806 09:00:17.772085 8511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:17.777735 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 8511
I20260806 09:00:17.778802 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:43403
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:34809
--webserver_port=35901
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:17.779873 8602 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:00:17.784648 8602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Bootstrap starting.
I20260806 09:00:17.805449 8602 log.cc:824] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:17.814703 8602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:00:17.815238 8602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Bootstrap complete.
I20260806 09:00:17.824601 8602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:17.825539 8602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608a1a3be79442cf89dd5a9919bbdf26, State: Initialized, Role: FOLLOWER
I20260806 09:00:17.826372 8602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:17.826669 8602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:17.826848 8602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:17.827020 8602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:00:17.831122 8602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:17.831693 8602 leader_election.cc:302] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 608a1a3be79442cf89dd5a9919bbdf26; no voters:
I20260806 09:00:17.832293 8602 leader_election.cc:288] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:00:17.832525 8610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:00:17.833475 8610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [term 2 LEADER]: Becoming Leader. State: Replica: 608a1a3be79442cf89dd5a9919bbdf26, State: Running, Role: LEADER
I20260806 09:00:17.834225 8610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } }
I20260806 09:00:17.834506 8602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:17.837110 8612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "608a1a3be79442cf89dd5a9919bbdf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } } }
I20260806 09:00:17.837658 8612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:17.837920 8613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608a1a3be79442cf89dd5a9919bbdf26. Latest consensus state: current_term: 2 leader_uuid: "608a1a3be79442cf89dd5a9919bbdf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608a1a3be79442cf89dd5a9919bbdf26" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34809 } } }
I20260806 09:00:17.838307 8613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:17.839705 8617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:17.846494 8617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:17.848222 8617 catalog_manager.cc:1259] Loaded cluster ID: a0b9864d014c462fa276f9c562ce4dc8
I20260806 09:00:17.848419 8617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:17.852916 8617 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:17.855075 8617 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Loaded TSK: 0
I20260806 09:00:17.857108 8617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:00:17.860057 8602 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 09:00:18.244767 8605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:18.245342 8605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:18.245563 8605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:18.257428 8605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:18.257680 8605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:18.257936 8605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:18.274163 8605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:43403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=35901
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:18.276180 8605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:18.278173 8605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:18.307469 8655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:18.307657 8653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:18.308933 8605 server_base.cc:1034] running on GCE node
W20260806 09:00:18.309477 8666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:18.310514 8605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:18.322533 8605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:18.323933 8605 hybrid_clock.cc:648] HybridClock initialized: now 1786006818323787 us; error 134 us; skew 500 ppm
I20260806 09:00:18.324404 8605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:18 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:18.330860 8605 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:18.335996 8605 webserver.cc:460] Webserver started at http://127.22.98.1:35901/ using document root <none> and password file <none>
I20260806 09:00:18.336966 8605 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:18.337131 8605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:00:18 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:18.377231 8605 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:00:18.380980 8678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:18.383299 8605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 09:00:18.383566 8605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "e721bcad07394d128942eabbb9a9b042"
format_stamp: "Formatted at 2026-08-06 09:00:14 on dist-test-slave-jpj1"
server_key: "fd7125e6f3e1742a4339f4456829ad52"
server_key_iv: "d778b3ec07ccf2a1a6f2cf565caa8e23"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:18.385133 8605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:18.405575 8605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:18.409685 8605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:18.409996 8605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:18.411137 8605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:18.412745 8605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:18.412846 8605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:18.412961 8605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:18.413040 8605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:18.483371 8605 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:43403
I20260806 09:00:18.483371 8843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:43403 every 8 connection(s)
I20260806 09:00:18.485145 8605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:18.489627 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 8605
I20260806 09:00:18.490514 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:34207
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:34809
--webserver_port=44623
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:18 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:18.540633 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:18.489814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.1:43771)
0806 09:00:18.498723 (+ 8909us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:18.498790 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:18.502890 (+ 4100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:18.516157 (+ 13267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:18.516173 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:18.516910 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:18.517564 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:18.517600 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:18.519467 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:18.519472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:18.520923 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:18.520943 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:18.522596 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:18.522605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:18.522919 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:18.523801 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:18.523841 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:18.526250 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:18.534723 (+ 8473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:18.534735 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:18.534755 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:18.535522 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:18.536295 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:18.536301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:18.536304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:18.536601 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:18.537681 (+ 1080us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:18.537690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:18.538125 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 09:00:18.538444 (+ 319us) client_negotiation.cc:240] Negotiation successful
0806 09:00:18.539051 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8523,"thread_start_us":10633,"threads_started":1}
I20260806 09:00:18.544219 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:18.489746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:43771 (local address 127.22.98.62:34809)
0806 09:00:18.502654 (+ 12908us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:18.502675 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:18.502739 (+ 64us) server_negotiation.cc:407] Connection header received
0806 09:00:18.503739 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:18.503762 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:18.506828 (+ 3066us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:18.515975 (+ 9147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:18.517866 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:18.519248 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:18.521141 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:18.522267 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:18.523490 (+ 1223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:18.527789 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:18.527802 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:18.527826 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:18.527884 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:18.533446 (+ 5562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:18.535802 (+ 2356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:18.535812 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:18.535821 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:18.536140 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:18.536768 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:18.536774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:18.536777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:18.537507 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:18.542260 (+ 4753us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:18.542595 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 09:00:18.543002 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12023,"spinlock_wait_cycles":1257728,"thread_start_us":11749,"threads_started":1}
I20260806 09:00:18.547487 8844 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:18.547894 8844 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:18.548859 8844 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
W20260806 09:00:18.552079 8844 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:34809 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 09:00:18.552321 8844 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:18.552611 8844 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:18.553927 8844 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:18.554205 8844 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
W20260806 09:00:18.555274 8844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:00:18.788551 8856 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 09:00:18.940757 8848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:18.941109 8848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:18.941257 8848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:18.948997 8848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:18.949139 8848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:18.949301 8848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:18.961051 8848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:34207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=44623
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:18.963059 8848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:18.964977 8848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:18.981827 8873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:18.982013 8875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:18.984788 8877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:18.985677 8848 server_base.cc:1034] running on GCE node
I20260806 09:00:18.986641 8848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:18.988087 8848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:18.989374 8848 hybrid_clock.cc:648] HybridClock initialized: now 1786006818989275 us; error 61 us; skew 500 ppm
I20260806 09:00:18.989825 8848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:18 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:18.997810 8848 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:19.000092 8848 webserver.cc:460] Webserver started at http://127.22.98.2:44623/ using document root <none> and password file <none>
I20260806 09:00:19.001114 8848 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:19.001287 8848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:19.028208 8848 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.000s
I20260806 09:00:19.049996 8889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:19.052182 8848 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.003s sys 0.018s
I20260806 09:00:19.052424 8848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "ea341218b50d49a2b38cd04139230bb2"
format_stamp: "Formatted at 2026-08-06 09:00:15 on dist-test-slave-jpj1"
server_key: "b3af806ef6dea29e66833fa5f58ed3db"
server_key_iv: "c42c44fe3ca2aff5e971e85c79350d13"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:19.053915 8848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:19.078552 8848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:19.084528 8848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:19.084946 8848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:19.092556 8848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:19.094589 8848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:19.094712 8848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:19.094866 8848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:19.094944 8848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:19.163932 8848 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:34207
I20260806 09:00:19.163982 9077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:34207 every 8 connection(s)
I20260806 09:00:19.165954 8848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:19.173125 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 8848
I20260806 09:00:19.173957 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.98.20:49889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:46357
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:34809
--webserver_port=34181
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:19.212396 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.169814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:54489 (local address 127.22.98.62:34809)
0806 09:00:19.170267 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.170271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.171616 (+ 1345us) server_negotiation.cc:407] Connection header received
0806 09:00:19.176631 (+ 5015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.176639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.176744 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.176849 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:19.192515 (+ 15666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.193645 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.196341 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.197012 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.197118 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.204398 (+ 7280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:19.204408 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:19.204413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:19.204464 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:19.208161 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.209277 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.209284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.209288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.209473 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.210031 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.210037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.210040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.210438 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:19.210607 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:19.211724 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0806 09:00:19.211988 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
I20260806 09:00:19.213771 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.170495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.2:54489)
0806 09:00:19.171414 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:19.171475 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:19.176178 (+ 4703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:19.190307 (+ 14129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:19.190322 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:19.191168 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:19.192214 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.192247 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.193902 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.193907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:19.195937 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.195956 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.197524 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.197530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:19.197831 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.198698 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:19.198727 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:19.204032 (+ 5305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:19.208391 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:19.208403 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:19.208426 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:19.209107 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:19.209602 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:19.209606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:19.209611 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:19.209897 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:19.210698 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:19.210708 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:19.211236 (+ 528us) client_negotiation.cc:769] Sending connection context
0806 09:00:19.211584 (+ 348us) client_negotiation.cc:240] Negotiation successful
0806 09:00:19.212326 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":372,"threads_started":1}
I20260806 09:00:19.215482 9078 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:19.215867 9078 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:19.216759 9078 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:19.219087 8553 ts_manager.cc:194] Registered new tserver with Master: ea341218b50d49a2b38cd04139230bb2 (127.22.98.2:34207)
I20260806 09:00:19.221410 8553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:54489
I20260806 09:00:19.555470 8844 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:19.555991 8844 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:19.557536 8553 ts_manager.cc:194] Registered new tserver with Master: e721bcad07394d128942eabbb9a9b042 (127.22.98.1:43403)
I20260806 09:00:19.558735 8553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:43771
W20260806 09:00:19.595052 9082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:19.595623 9082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:19.595849 9082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:19.607630 9082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:19.607856 9082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:19.608094 9082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:19.624467 9082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.98.20:49889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:46357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=34181
--tserver_master_addrs=127.22.98.62:34809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:19.626585 9082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:19.628572 9082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:19.697548 9096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:19.707351 9095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:19.707705 9101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:19.710268 9082 server_base.cc:1034] running on GCE node
I20260806 09:00:19.711031 9082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:19.712231 9082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:19.713532 9082 hybrid_clock.cc:648] HybridClock initialized: now 1786006819713462 us; error 66 us; skew 500 ppm
I20260806 09:00:19.713879 9082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:19.717293 9082 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:19.720271 9082 webserver.cc:460] Webserver started at http://127.22.98.3:34181/ using document root <none> and password file <none>
I20260806 09:00:19.721223 9082 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:19.721382 9082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:19.743842 9082 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 09:00:19.747025 9111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:19.748726 9082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:00:19.748932 9082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "b5e5491f93ed457eaaa56eacade8999a"
format_stamp: "Formatted at 2026-08-06 09:00:16 on dist-test-slave-jpj1"
server_key: "5a263999cf9dab3413c295124a0c9444"
server_key_iv: "edba376a0795a5f8d81820d6fc0ba25e"
server_key_version: "kuduclusterkey@0"
I20260806 09:00:19.750026 9082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:19.770330 9082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:19.775832 9082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:19.776234 9082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:19.782581 9082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:19.786916 9082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:19.787058 9082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:19.787206 9082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:19.787297 9082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:19.839205 9082 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:46357
I20260806 09:00:19.839262 9272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:46357 every 8 connection(s)
I20260806 09:00:19.841295 9082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:19.849946 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9082
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:19.882753 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.844260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:33131 (local address 127.22.98.62:34809)
0806 09:00:19.850343 (+ 6083us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.850352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.857944 (+ 7592us) server_negotiation.cc:407] Connection header received
0806 09:00:19.863822 (+ 5878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.863829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.863942 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.864076 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:19.866278 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.867349 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.869785 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.870381 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.870472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.875236 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:19.875243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:19.875250 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:19.875306 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:19.878621 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.879655 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.879661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.879664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.879821 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.880423 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.880428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.880432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.880791 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:19.880925 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:19.882065 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0806 09:00:19.882322 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5925,"thread_start_us":6617,"threads_started":1}
I20260806 09:00:19.885628 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.845048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34809 (local address 127.22.98.3:33131)
0806 09:00:19.857719 (+ 12671us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:19.857789 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:19.863234 (+ 5445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:19.864403 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:19.864418 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:19.865138 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:19.866024 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.866051 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.867611 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.867616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:19.869621 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.869636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.870870 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.870877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:19.871171 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.872216 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:19.872246 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:19.874941 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:19.878807 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:19.878818 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:19.878839 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:19.879475 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:19.879974 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:19.879979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:19.879982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:19.880287 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:19.881026 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:19.881034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:19.881563 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 09:00:19.883319 (+ 1756us) client_negotiation.cc:240] Negotiation successful
0806 09:00:19.883997 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12315,"thread_start_us":18394,"threads_started":1}
I20260806 09:00:19.886897 9283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.851294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.22.98.62:34809)
0806 09:00:19.863374 (+ 12080us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.863381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.867512 (+ 4131us) server_negotiation.cc:407] Connection header received
0806 09:00:19.868061 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.868067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.868186 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.868268 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:19.870578 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.871785 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.874705 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.875390 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.875493 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.879667 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:19.879674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:19.879679 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:19.879721 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:19.883360 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.884366 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.884372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.884375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.884536 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.885028 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.885033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.885036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.885371 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:19.885640 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:19.886263 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 09:00:19.886502 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11897,"thread_start_us":12601,"threads_started":1}
I20260806 09:00:19.887775 9274 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34809
I20260806 09:00:19.888129 9274 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:19.888907 9274 heartbeater.cc:511] Master 127.22.98.62:34809 requested a full tablet report, sending...
I20260806 09:00:19.890508 8553 ts_manager.cc:194] Registered new tserver with Master: b5e5491f93ed457eaaa56eacade8999a (127.22.98.3:46357)
I20260806 09:00:19.891441 8553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:33131
I20260806 09:00:19.894855 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:00:19.911576 9283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.898222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.22.98.62:34809)
0806 09:00:19.898438 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.898446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.898957 (+ 511us) server_negotiation.cc:407] Connection header received
0806 09:00:19.899135 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.899141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.899223 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.899304 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:19.900891 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.901583 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.902972 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.903424 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.903584 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:19.906137 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:19.906142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:19.906145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:19.906183 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:19.908749 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.909488 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.909492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.909495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.909638 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:19.910099 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:19.910105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:19.910107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:19.910385 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:19.910617 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:19.910967 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 09:00:19.911213 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 09:00:19.916713 8553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:00:19.927131 8553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 06 09:00:19 dist-test-slave-jpj1 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for HTTP/127.22.98.20@KRBTEST.COM
I20260806 09:00:20.004056 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.968137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.22.98.1:43403)
0806 09:00:19.980142 (+ 12005us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.980153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.990911 (+ 10758us) server_negotiation.cc:407] Connection header received
0806 09:00:19.991147 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.991157 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.991479 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.991817 (+ 338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:19.993477 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.995090 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.000591 (+ 5501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.002129 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.002264 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.002525 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.003484 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.003719 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11753,"thread_start_us":11770,"threads_started":1}
I20260806 09:00:20.005380 9301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.967891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:46612)
0806 09:00:19.990741 (+ 22850us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:19.990784 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:19.990945 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:19.992162 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:19.992170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:19.992205 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:19.993214 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.993232 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.995316 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.995322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:19.997911 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.997928 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.002927 (+ 4999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.002935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.003165 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.003241 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 09:00:20.004627 (+ 1386us) client_negotiation.cc:240] Negotiation successful
0806 09:00:20.004870 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22720,"mutex_wait_us":1,"thread_start_us":22539,"threads_started":1}
I20260806 09:00:20.005793 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.973661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:34207 (local address 127.0.0.1:33722)
0806 09:00:19.997046 (+ 23385us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:19.997066 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:19.997188 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:19.998237 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:19.998242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:19.998272 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:19.998709 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.998725 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.000794 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.000800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.002809 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.002831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.004976 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.004983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.005168 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.005206 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:00:20.005310 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 09:00:20.005458 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23246,"thread_start_us":14201,"threads_started":1}
I20260806 09:00:20.006991 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.956682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:46357 (local address 127.0.0.1:35866)
0806 09:00:19.988383 (+ 31701us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:19.988438 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:19.988659 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:19.994231 (+ 5572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:19.994238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:19.994271 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:19.994759 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:19.994782 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.997231 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:19.997236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.000347 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.000362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.006162 (+ 5800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.006170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.006337 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.006370 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:00:20.006462 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:00:20.006615 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31349,"mutex_wait_us":304,"thread_start_us":38441,"threads_started":1}
I20260806 09:00:20.008198 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.974306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.22.98.2:34207)
0806 09:00:19.989331 (+ 15025us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.989341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.997335 (+ 7994us) server_negotiation.cc:407] Connection header received
0806 09:00:19.997459 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.997468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.997754 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.997981 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:19.998959 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.000591 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.003017 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.004464 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.004589 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.004792 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.006062 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.007882 (+ 1820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14803,"thread_start_us":14667,"threads_started":1}
I20260806 09:00:20.010423 8753 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 (DEFAULT_TABLE table=test-table [id=98147942347d469dbccd87adc6e94438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:20.012223 8753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf47a1eddcb4af38cc3e93644ce0397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:20.012612 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:19.957291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.22.98.3:46357)
0806 09:00:19.993185 (+ 35894us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:19.993195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:19.993236 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:00:19.993346 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:19.993355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:19.993646 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:19.993876 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:19.995449 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:19.997025 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.001531 (+ 4506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.002993 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.003126 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.003369 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.012021 (+ 8652us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.012235 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35704,"thread_start_us":35558,"threads_started":1}
I20260806 09:00:20.012440 8965 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 (DEFAULT_TABLE table=test-table [id=98147942347d469dbccd87adc6e94438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:20.016623 9176 tablet_service.cc:1467] Processing CreateTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 (DEFAULT_TABLE table=test-table [id=98147942347d469dbccd87adc6e94438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:20.018690 9176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf47a1eddcb4af38cc3e93644ce0397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:20.019778 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bf47a1eddcb4af38cc3e93644ce0397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:20.062973 9306 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Bootstrap starting.
I20260806 09:00:20.072259 9306 tablet_bootstrap.cc:654] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:20.073256 9307 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Bootstrap starting.
I20260806 09:00:20.073885 9306 log.cc:824] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:20.079862 9307 tablet_bootstrap.cc:654] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:20.081449 9307 log.cc:824] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:20.091830 9309 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Bootstrap starting.
I20260806 09:00:20.102010 9309 tablet_bootstrap.cc:654] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:20.107834 9309 log.cc:824] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:20.113040 9307 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: No bootstrap required, opened a new log
I20260806 09:00:20.113464 9307 ts_tablet_manager.cc:1397] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Time spent bootstrapping tablet: real 0.041s user 0.025s sys 0.011s
I20260806 09:00:20.113508 9306 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: No bootstrap required, opened a new log
I20260806 09:00:20.113780 9306 ts_tablet_manager.cc:1397] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Time spent bootstrapping tablet: real 0.054s user 0.014s sys 0.027s
I20260806 09:00:20.124507 9306 raft_consensus.cc:348] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.124507 9307 raft_consensus.cc:348] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.125056 9307 raft_consensus.cc:374] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:20.125090 9306 raft_consensus.cc:374] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:20.125195 9306 raft_consensus.cc:729] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea341218b50d49a2b38cd04139230bb2, State: Initialized, Role: FOLLOWER
I20260806 09:00:20.125272 9307 raft_consensus.cc:729] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e721bcad07394d128942eabbb9a9b042, State: Initialized, Role: FOLLOWER
I20260806 09:00:20.126075 9307 consensus_queue.cc:260] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.126075 9306 consensus_queue.cc:260] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.131529 9307 ts_tablet_manager.cc:1428] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260806 09:00:20.132051 8844 heartbeater.cc:503] Master 127.22.98.62:34809 was elected leader, sending a full tablet report...
I20260806 09:00:20.138607 9309 tablet_bootstrap.cc:492] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: No bootstrap required, opened a new log
I20260806 09:00:20.140933 9078 heartbeater.cc:503] Master 127.22.98.62:34809 was elected leader, sending a full tablet report...
I20260806 09:00:20.141885 9306 ts_tablet_manager.cc:1428] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Time spent starting tablet: real 0.028s user 0.016s sys 0.013s
I20260806 09:00:20.142627 9309 ts_tablet_manager.cc:1397] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.006s
I20260806 09:00:20.153462 9309 raft_consensus.cc:348] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.154105 9309 raft_consensus.cc:374] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:20.154268 9309 raft_consensus.cc:729] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e5491f93ed457eaaa56eacade8999a, State: Initialized, Role: FOLLOWER
I20260806 09:00:20.155023 9309 consensus_queue.cc:260] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
W20260806 09:00:20.170392 9079 tablet.cc:2369] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:20.178164 9315 raft_consensus.cc:482] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:20.178316 9274 heartbeater.cc:503] Master 127.22.98.62:34809 was elected leader, sending a full tablet report...
I20260806 09:00:20.178493 9315 raft_consensus.cc:504] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.179083 9309 ts_tablet_manager.cc:1428] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Time spent starting tablet: real 0.036s user 0.019s sys 0.015s
I20260806 09:00:20.183962 9315 leader_election.cc:288] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e721bcad07394d128942eabbb9a9b042 (127.22.98.1:43403), ea341218b50d49a2b38cd04139230bb2 (127.22.98.2:34207)
I20260806 09:00:20.196487 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.185217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:45325 (local address 127.22.98.2:34207)
0806 09:00:20.185421 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.185427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.185450 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:00:20.185529 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.185533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.185622 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.185729 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:20.187806 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.188861 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.191461 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.192897 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.193055 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.193178 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.196038 (+ 2860us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.196201 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:00:20.195886 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.184211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:34207 (local address 127.22.98.3:45325)
0806 09:00:20.184711 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.184733 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:20.184879 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:20.185914 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:20.185920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:20.185949 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:20.186169 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.186181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.189047 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.189052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.191173 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.191194 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.193369 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.193375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.193597 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.195289 (+ 1692us) client_negotiation.cc:769] Sending connection context
0806 09:00:20.195379 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 09:00:20.195557 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353}
I20260806 09:00:20.198060 9007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf47a1eddcb4af38cc3e93644ce0397" candidate_uuid: "b5e5491f93ed457eaaa56eacade8999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea341218b50d49a2b38cd04139230bb2" is_pre_election: true
I20260806 09:00:20.198714 9007 raft_consensus.cc:2454] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e5491f93ed457eaaa56eacade8999a in term 0.
I20260806 09:00:20.199990 9114 leader_election.cc:302] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5e5491f93ed457eaaa56eacade8999a, ea341218b50d49a2b38cd04139230bb2; no voters:
I20260806 09:00:20.200637 9315 raft_consensus.cc:2781] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:00:20.200804 9315 raft_consensus.cc:482] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:00:20.200902 9315 raft_consensus.cc:3037] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:20.204497 9315 raft_consensus.cc:504] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.205940 9007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf47a1eddcb4af38cc3e93644ce0397" candidate_uuid: "b5e5491f93ed457eaaa56eacade8999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea341218b50d49a2b38cd04139230bb2"
I20260806 09:00:20.206169 9007 raft_consensus.cc:3037] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:20.207921 9315 leader_election.cc:288] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [CANDIDATE]: Term 1 election: Requested vote from peers e721bcad07394d128942eabbb9a9b042 (127.22.98.1:43403), ea341218b50d49a2b38cd04139230bb2 (127.22.98.2:34207)
I20260806 09:00:20.209591 9007 raft_consensus.cc:2454] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e5491f93ed457eaaa56eacade8999a in term 1.
I20260806 09:00:20.210158 9114 leader_election.cc:302] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5e5491f93ed457eaaa56eacade8999a, ea341218b50d49a2b38cd04139230bb2; no voters:
I20260806 09:00:20.210637 9315 raft_consensus.cc:2781] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:20.210734 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.185212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:37877 (local address 127.22.98.1:43403)
0806 09:00:20.185492 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.185499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.197248 (+ 11749us) server_negotiation.cc:407] Connection header received
0806 09:00:20.197541 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.197548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.197666 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.197737 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:20.198338 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.199341 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.208159 (+ 8818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.209414 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.209546 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.209697 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.210323 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.210470 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 09:00:20.211002 9315 raft_consensus.cc:686] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 1 LEADER]: Becoming Leader. State: Replica: b5e5491f93ed457eaaa56eacade8999a, State: Running, Role: LEADER
I20260806 09:00:20.211457 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.184338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.22.98.3:37877)
0806 09:00:20.197149 (+ 12811us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.197169 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:20.197367 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:20.197943 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:20.197947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:20.197972 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:20.198158 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.198169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.199565 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.199570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.207708 (+ 8138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:20.207730 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.209880 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.209886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:20.210109 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.210150 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 09:00:20.211023 (+ 873us) client_negotiation.cc:240] Negotiation successful
0806 09:00:20.211172 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12707,"threads_started":1}
I20260806 09:00:20.211701 9315 consensus_queue.cc:237] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } }
I20260806 09:00:20.219399 8784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf47a1eddcb4af38cc3e93644ce0397" candidate_uuid: "b5e5491f93ed457eaaa56eacade8999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e721bcad07394d128942eabbb9a9b042" is_pre_election: true
I20260806 09:00:20.220000 8782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bf47a1eddcb4af38cc3e93644ce0397" candidate_uuid: "b5e5491f93ed457eaaa56eacade8999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e721bcad07394d128942eabbb9a9b042"
I20260806 09:00:20.220111 8784 raft_consensus.cc:2454] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e5491f93ed457eaaa56eacade8999a in term 0.
I20260806 09:00:20.229534 8553 catalog_manager.cc:5495] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a reported cstate change: term changed from 0 to 1, leader changed from <none> to b5e5491f93ed457eaaa56eacade8999a (127.22.98.3). New cstate: current_term: 1 leader_uuid: "b5e5491f93ed457eaaa56eacade8999a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e5491f93ed457eaaa56eacade8999a" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 46357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 } health_report { overall_health: UNKNOWN } } }
W20260806 09:00:20.255604 8845 tablet.cc:2369] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:20.278848 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.270326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.22.98.3:46357)
0806 09:00:20.270528 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.270534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.270556 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:00:20.270696 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.270702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.270835 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.271022 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:00:20.271745 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.272870 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.275090 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.275792 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.275934 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.276769 (+ 835us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:00:20.277576 (+ 807us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:00:20.277741 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.278454 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.278553 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:00:20.289316 9007 raft_consensus.cc:1261] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 1 FOLLOWER]: Refusing update from remote peer b5e5491f93ed457eaaa56eacade8999a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:20.290139 8782 raft_consensus.cc:3037] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:20.290725 9315 consensus_queue.cc:1035] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea341218b50d49a2b38cd04139230bb2" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:20.293902 8782 raft_consensus.cc:1261] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 1 FOLLOWER]: Refusing update from remote peer b5e5491f93ed457eaaa56eacade8999a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:20.294905 9315 consensus_queue.cc:1035] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e721bcad07394d128942eabbb9a9b042" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:20.319316 9322 mvcc.cc:205] Tried to move back new op lower bound from 7315483935891664896 to 7315483935593750528. Current Snapshot: MvccSnapshot[applied={T|T < 7315483935891664896}]
I20260806 09:00:20.357854 8551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:20.358243 8551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:20.363209 8551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 608a1a3be79442cf89dd5a9919bbdf26: Sending DeleteTablet for 3 replicas of tablet 4bf47a1eddcb4af38cc3e93644ce0397
I20260806 09:00:20.366753 9176 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:20 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35866
I20260806 09:00:20.367854 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:20 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:33722
I20260806 09:00:20.368968 8753 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bf47a1eddcb4af38cc3e93644ce0397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:20 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:46612
I20260806 09:00:20.372387 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 8605
I20260806 09:00:20.403349 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.374809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.22.98.1:43403)
0806 09:00:20.375006 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.375013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.375034 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:00:20.375130 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.375137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.375242 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.375348 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:20.382922 (+ 7574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.384230 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.386742 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.387448 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.387623 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.392081 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:20.392105 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:20.392131 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:20.392192 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:20.396822 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.397961 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.397974 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.397978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.398187 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.398692 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.398697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.398700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.399255 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:20.399640 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.402626 (+ 2986us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.402905 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:00:20.408491 8838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:20.408769 9336 tablet_replica.cc:330] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: stopping tablet replica
I20260806 09:00:20.409389 9336 raft_consensus.cc:2229] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:00:20.410113 9338 tablet_replica.cc:330] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: stopping tablet replica
I20260806 09:00:20.410899 9336 raft_consensus.cc:2258] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:20.411515 9338 raft_consensus.cc:2229] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:20.412802 9338 raft_consensus.cc:2258] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:20.415123 9336 ts_tablet_manager.cc:1905] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:20.417971 9338 ts_tablet_manager.cc:1905] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:20.436066 9336 ts_tablet_manager.cc:1918] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:20.436743 9336 log.cc:1196] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/4bf47a1eddcb4af38cc3e93644ce0397
I20260806 09:00:20.436890 9338 ts_tablet_manager.cc:1918] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:20.437247 9338 log.cc:1196] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/4bf47a1eddcb4af38cc3e93644ce0397
I20260806 09:00:20.437938 9338 ts_tablet_manager.cc:1939] T 4bf47a1eddcb4af38cc3e93644ce0397 P ea341218b50d49a2b38cd04139230bb2: Deleting consensus metadata
I20260806 09:00:20.440388 8535 catalog_manager.cc:4841] TS ea341218b50d49a2b38cd04139230bb2 (127.22.98.2:34207): tablet 4bf47a1eddcb4af38cc3e93644ce0397 (table test-table [id=98147942347d469dbccd87adc6e94438]) successfully deleted
I20260806 09:00:20.441314 9336 ts_tablet_manager.cc:1939] T 4bf47a1eddcb4af38cc3e93644ce0397 P b5e5491f93ed457eaaa56eacade8999a: Deleting consensus metadata
I20260806 09:00:20.443442 8537 catalog_manager.cc:4841] TS b5e5491f93ed457eaaa56eacade8999a (127.22.98.3:46357): tablet 4bf47a1eddcb4af38cc3e93644ce0397 (table test-table [id=98147942347d469dbccd87adc6e94438]) successfully deleted
I20260806 09:00:20.603857 9339 tablet_replica.cc:330] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042: stopping tablet replica
I20260806 09:00:20.604414 9339 raft_consensus.cc:2229] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:20.604982 9339 raft_consensus.cc:2258] T 4bf47a1eddcb4af38cc3e93644ce0397 P e721bcad07394d128942eabbb9a9b042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:20.605894 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 8605
W20260806 09:00:20.639183 8534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:43403 (error 108)
I20260806 09:00:20.639308 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 8848
I20260806 09:00:20.661657 9305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.640446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.22.98.2:34207)
0806 09:00:20.641338 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.641345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.641367 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:00:20.641448 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.641453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.641555 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.641666 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:20.644070 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.645162 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.647147 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.647831 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.647971 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.652278 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:20.652299 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:20.652321 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:20.652371 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:20.656991 (+ 4620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.658024 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.658034 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.658038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.658242 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.658723 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.658728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.658731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.659284 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:20.659674 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.660930 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.661263 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742}
I20260806 09:00:20.663038 9070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:20.680514 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.655361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:46630)
0806 09:00:20.680027 (+ 24666us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.680239 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24479,"thread_start_us":24379,"threads_started":1}
W20260806 09:00:20.680959 8534 catalog_manager.cc:4584] TS e721bcad07394d128942eabbb9a9b042 (127.22.98.1:43403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4bf47a1eddcb4af38cc3e93644ce0397: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
I20260806 09:00:20.735016 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.734443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:46646)
0806 09:00:20.734631 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.734813 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 09:00:20.736563 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.736153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57672)
0806 09:00:20.736299 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.736380 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 09:00:20.797340 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.796825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57674)
0806 09:00:20.797065 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.797158 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 09:00:20.798471 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.798092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57680)
0806 09:00:20.798252 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.798328 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 09:00:20.816990 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 8848
I20260806 09:00:20.842103 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9082
I20260806 09:00:20.861968 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.842876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.22.98.3:46357)
0806 09:00:20.847475 (+ 4599us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:20.847480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:20.847500 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:00:20.847581 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:20.847586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:20.847660 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:20.847785 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:20.849275 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.850041 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.851780 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:20.852238 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:20.852354 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:20.855315 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:20.855327 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:20.855345 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:20.855384 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:20.858489 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.859219 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.859227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.859230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.859376 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:20.859928 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:20.859933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:20.859935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:20.860373 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:20.860721 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:20.861042 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 09:00:20.861311 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4472,"thread_start_us":4373,"threads_started":1}
I20260806 09:00:20.862874 9266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:20.879324 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.878750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57690)
0806 09:00:20.878996 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.879106 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 09:00:20.880543 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:20.880058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57706)
0806 09:00:20.880278 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:20.880373 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 09:00:20.985589 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9082
I20260806 09:00:21.014132 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 8511
I20260806 09:00:21.015580 8596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:21.129595 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:21.129048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43403 (local address 127.0.0.1:57718)
0806 09:00:21.129277 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:21.129384 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:43403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 09:00:21.129679 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 8511
I20260806 09:00:21.159816 22920 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 09:00:21.505968 22920 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 09:00:21.506238 22920 mini_ranger.cc:66] Stopping Ranger...
I20260806 09:00:21.852205 22920 mini_ranger.cc:68] Stopped Ranger
2026-08-06 09:00:21.852 UTC [7161] LOG: received smart shutdown request
2026-08-06 09:00:21.855 UTC [7161] LOG: background worker "logical replication launcher" (PID 7173) exited with exit code 1
2026-08-06 09:00:21.855 UTC [7167] LOG: shutting down
2026-08-06 09:00:22.001 UTC [7161] LOG: database system is shut down
2026-08-06T09:00:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54378 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:00:22 dist-test-slave-jpj1 krb5kdc[9370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:00:22 dist-test-slave-jpj1 krb5kdc[9370](info): set up 2 sockets
Aug 06 09:00:22 dist-test-slave-jpj1 krb5kdc[9370](info): commencing operation
krb5kdc: starting...
W20260806 09:00:23.316031 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:00:23 dist-test-slave-jpj1 krb5kdc[9370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:00:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:00:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:00:23.607175 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42539
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:23.960892 9389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:23.961187 9389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:23.961321 9389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:23.968719 9389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:23.968835 9389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:23.968894 9389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:23.968973 9389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:23.969022 9389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:23.979135 9389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:42539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:23.980592 9389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:23.982044 9389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:23.994513 9399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:23.994432 9396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:23.994464 9395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:23.995747 9389 server_base.cc:1034] running on GCE node
I20260806 09:00:23.996682 9389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:23.998183 9389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:23.999402 9389 hybrid_clock.cc:648] HybridClock initialized: now 1786006823999348 us; error 47 us; skew 500 ppm
I20260806 09:00:23.999804 9389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:24.006469 9389 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:24.008206 9389 webserver.cc:460] Webserver started at http://127.22.98.62:35169/ using document root <none> and password file <none>
I20260806 09:00:24.008944 9389 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:24.009073 9389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:24.009397 9389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:24.012624 9389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "1a614776d6b14ba6b75c5cceb06991b6"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "f06aba4cba20c79a120ae26b41072f5b"
server_key_iv: "1fb0cfe0a2ca939f100c1aceaf6fc59a"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.013451 9389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "1a614776d6b14ba6b75c5cceb06991b6"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "f06aba4cba20c79a120ae26b41072f5b"
server_key_iv: "1fb0cfe0a2ca939f100c1aceaf6fc59a"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.018895 9389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:00:24.022567 9409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:24.024077 9389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:24.024257 9389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "1a614776d6b14ba6b75c5cceb06991b6"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "f06aba4cba20c79a120ae26b41072f5b"
server_key_iv: "1fb0cfe0a2ca939f100c1aceaf6fc59a"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.024474 9389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:24.068202 9389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:24.072921 9389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:24.073211 9389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:24.092897 9389 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:42539
I20260806 09:00:24.092959 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:42539 every 8 connection(s)
I20260806 09:00:24.094451 9389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:24.096549 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9389
I20260806 09:00:24.096818 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:00:24.097513 22920 external_mini_cluster.cc:1428] Setting key da409066900aedb03820c8416b2d0571
I20260806 09:00:24.098883 9485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:24.106611 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.98.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.61@KRBTEST.COM" created.
I20260806 09:00:24.131273 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.110420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42539 (local address 127.0.0.1:46832)
0806 09:00:24.111431 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.111495 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:24.113646 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:24.114541 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:24.114548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:24.115297 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:24.115717 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.115740 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.117327 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.117331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.118451 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.118466 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.119459 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.119467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.119667 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.120415 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:24.120440 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:24.122977 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:24.126675 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.126688 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.126699 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.127120 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.127765 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.127769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.127772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.128044 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.129041 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:24.129050 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:24.129340 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 09:00:24.129676 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 09:00:24.130116 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":435,"threads_started":1}
I20260806 09:00:24.131309 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.110421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.22.98.62:42539)
0806 09:00:24.112367 (+ 1946us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:24.112376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:24.112433 (+ 57us) server_negotiation.cc:407] Connection header received
0806 09:00:24.114039 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:24.114046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:24.114300 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:24.114499 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:24.115924 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.117164 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.118599 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.119198 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.119384 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.123255 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:24.123265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:24.123280 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:24.123320 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:24.126500 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.127316 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.127325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.127329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.127573 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.128166 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.128172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.128175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.128874 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:24.129086 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:24.129853 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 09:00:24.130301 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":1709,"spinlock_wait_cycles":22784,"thread_start_us":1472,"threads_started":1}
I20260806 09:00:24.134586 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:00:24.135943 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.135412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53396)
0806 09:00:24.135660 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.135819 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 09:00:24.136243 9411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
I20260806 09:00:24.137873 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.137525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53398)
0806 09:00:24.137665 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.137744 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 09:00:24.138085 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34137: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
I20260806 09:00:24.159614 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:34137
--webserver_interface=127.22.98.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:24.187633 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } attempt: 1
I20260806 09:00:24.188983 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.188419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53402)
0806 09:00:24.188643 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.188736 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 09:00:24.190068 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.189700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53408)
0806 09:00:24.189853 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.189930 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 09:00:24.190279 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34137: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
I20260806 09:00:24.258561 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } attempt: 2
I20260806 09:00:24.260011 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.259407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53424)
0806 09:00:24.259744 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.259851 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 09:00:24.261242 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.260781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53428)
0806 09:00:24.260974 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.261057 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 09:00:24.261484 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34137: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
I20260806 09:00:24.352751 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } attempt: 3
I20260806 09:00:24.353976 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.353512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53442)
0806 09:00:24.353737 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.353834 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 09:00:24.355018 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.354648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53458)
0806 09:00:24.354786 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.354880 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:00:24.355221 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34137: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
I20260806 09:00:24.498483 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } attempt: 4
I20260806 09:00:24.499708 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.499130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53460)
0806 09:00:24.499429 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.499567 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260806 09:00:24.500898 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.500521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53464)
0806 09:00:24.500680 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.500777 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 09:00:24.501107 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34137: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34137: connect: Connection refused (error 111)
W20260806 09:00:24.512765 9495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:24.513082 9495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:24.513216 9495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:24.520808 9495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:24.520931 9495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:24.520989 9495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:24.521075 9495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:24.521139 9495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:24.531490 9495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.98.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:24.532970 9495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:24.534305 9495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:24.546984 9502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:24.547058 9507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:24.547031 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:24.547525 9495 server_base.cc:1034] running on GCE node
I20260806 09:00:24.548503 9495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:24.550212 9495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:24.551441 9495 hybrid_clock.cc:648] HybridClock initialized: now 1786006824551385 us; error 57 us; skew 500 ppm
I20260806 09:00:24.551851 9495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:24.555152 9495 init.cc:377] Logged in from keytab as kudu/127.22.98.61@KRBTEST.COM (short username kudu)
I20260806 09:00:24.556761 9495 webserver.cc:460] Webserver started at http://127.22.98.61:37863/ using document root <none> and password file <none>
I20260806 09:00:24.557489 9495 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:24.557613 9495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:24.557920 9495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:24.561098 9495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data/instance:
uuid: "4d90e2041dfd4083b55e841612435616"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "ff99e1e9e7243231a24e1868915f63c7"
server_key_iv: "5f54b383d6eb8c732c41061e91699430"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.562098 9495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal/instance:
uuid: "4d90e2041dfd4083b55e841612435616"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "ff99e1e9e7243231a24e1868915f63c7"
server_key_iv: "5f54b383d6eb8c732c41061e91699430"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.567406 9495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260806 09:00:24.571060 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:24.572629 9495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:24.572816 9495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "4d90e2041dfd4083b55e841612435616"
format_stamp: "Formatted at 2026-08-06 09:00:24 on dist-test-slave-jpj1"
server_key: "ff99e1e9e7243231a24e1868915f63c7"
server_key_iv: "5f54b383d6eb8c732c41061e91699430"
server_key_version: "encryptionkey@0"
I20260806 09:00:24.573014 9495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:24.588340 9495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:24.593062 9495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:24.593358 9495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:24.611374 9495 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.61:34137
I20260806 09:00:24.611409 9596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.61:34137 every 8 connection(s)
I20260806 09:00:24.612833 9495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
I20260806 09:00:24.616932 9598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:24.617304 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9495
I20260806 09:00:24.617488 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-1/wal/instance
I20260806 09:00:24.617961 22920 external_mini_cluster.cc:1428] Setting key d5b3cbc3cd0e181b88643242bb7549ed
I20260806 09:00:24.624173 9598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.98.60@KRBTEST.COM; defaulting to no policy
I20260806 09:00:24.648919 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.627210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.22.98.62:42539)
0806 09:00:24.627445 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:24.627452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:24.628923 (+ 1471us) server_negotiation.cc:407] Connection header received
0806 09:00:24.631570 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:24.631579 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:24.631667 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:24.631755 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:24.634026 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.634703 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.637043 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.637496 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.637608 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.642388 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:24.642396 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:24.642401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:24.642443 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:24.645263 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.646317 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.646323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.646326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.646489 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.646950 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.646955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.646957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.647333 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:24.647464 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:24.648418 (+ 954us) server_negotiation.cc:299] Negotiation successful
0806 09:00:24.648629 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.60@KRBTEST.COM" created.
I20260806 09:00:24.649928 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.627796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42539 (local address 127.0.0.1:46842)
0806 09:00:24.628727 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.628779 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:24.631176 (+ 2397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:24.632065 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:24.632080 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:24.632921 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:24.633807 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.633837 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.634890 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.634895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.636856 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.636873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.637753 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.637760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.638083 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.638896 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:24.638925 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:24.642016 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:24.645447 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.645459 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.645481 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.646146 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.646616 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.646620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.646622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.646819 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.647474 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:24.647481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:24.647976 (+ 495us) client_negotiation.cc:769] Sending connection context
0806 09:00:24.648236 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 09:00:24.648647 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":399,"threads_started":1}
I20260806 09:00:24.651717 9598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:00:24.668622 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.652542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53466)
0806 09:00:24.652840 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.652859 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:24.652961 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:24.654084 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:24.654089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:24.654519 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:24.654687 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.654697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.656482 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.656487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.657554 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.657567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.658605 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.658610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.658811 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.659438 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:24.659459 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:24.661971 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:24.665549 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.665555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.665559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.666141 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.666627 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.666632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.666636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.666833 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.667591 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:24.667596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:24.667753 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 09:00:24.667902 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 09:00:24.668195 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260806 09:00:24.668968 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.652611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.22.98.61:34137)
0806 09:00:24.653563 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:24.653571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:24.653605 (+ 34us) server_negotiation.cc:407] Connection header received
0806 09:00:24.653661 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:24.653670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:24.653913 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:24.654060 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:24.654858 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.656352 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.657704 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.658463 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.658581 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.662240 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:24.662253 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:24.662270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:24.662309 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:24.665345 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.666328 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.666337 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.666340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.666496 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.666959 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.666963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.666965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.667395 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:24.667663 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:24.668153 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 09:00:24.668376 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":560,"threads_started":1}
I20260806 09:00:24.671952 9598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:00:24.673271 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.672685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55774)
0806 09:00:24.672898 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.673066 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 09:00:24.673545 9523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.675338 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.674902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55782)
0806 09:00:24.675077 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.675182 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 09:00:24.675619 9598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
I20260806 09:00:24.688050 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:37153
--webserver_interface=127.22.98.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:24.724992 9598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 1
I20260806 09:00:24.726320 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.725762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55794)
0806 09:00:24.725984 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.726116 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 09:00:24.727409 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.726993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55798)
0806 09:00:24.727134 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.727235 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 09:00:24.727671 9598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.795905 9598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 2
I20260806 09:00:24.797219 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.796645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55804)
0806 09:00:24.796901 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.797017 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 09:00:24.798396 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.797989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55808)
0806 09:00:24.798146 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.798237 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:00:24.798643 9598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.800374 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } attempt: 5
Aug 06 09:00:24 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:00:24.813688 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.801023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53472)
0806 09:00:24.801238 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.801258 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:24.801392 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:24.801733 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:24.801738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:24.802169 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:24.802362 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.802371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.803356 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.803360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.804292 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:24.804305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.805049 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.805056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:24.805236 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.805829 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:24.805849 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:24.807955 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:24.811190 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.811196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.811200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.811660 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.812023 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:24.812029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:24.812032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:24.812178 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:24.812714 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:24.812719 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:24.812880 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 09:00:24.813001 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 09:00:24.813308 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 09:00:24.813699 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.801064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.22.98.61:34137)
0806 09:00:24.801229 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:24.801235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:24.801325 (+ 90us) server_negotiation.cc:407] Connection header received
0806 09:00:24.801509 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:24.801514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:24.801594 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:24.801686 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:24.802515 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.803210 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.804433 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:24.804899 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:24.805016 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:24.808153 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:24.808158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:24.808162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:24.808203 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:24.811021 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.811781 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.811787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.811789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.811915 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:24.812293 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:24.812296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:24.812298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:24.812596 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:24.812746 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:24.813154 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 09:00:24.813377 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:00:24.814680 9485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:00:24.815696 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.815265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55820)
0806 09:00:24.815420 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.815500 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 09:00:24.816695 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.816359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55822)
0806 09:00:24.816507 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.816591 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:00:24.816897 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.868144 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 1
I20260806 09:00:24.869364 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.868857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55830)
0806 09:00:24.869072 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.869171 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 09:00:24.870465 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.870112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55846)
0806 09:00:24.870269 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.870357 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:00:24.870713 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.889911 9598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 3
I20260806 09:00:24.891245 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.890646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55850)
0806 09:00:24.890912 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.891042 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 09:00:24.892462 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.891947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55854)
0806 09:00:24.892201 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.892307 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260806 09:00:24.892732 9598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:24.938949 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 2
I20260806 09:00:24.940142 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.939632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55856)
0806 09:00:24.939871 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.939962 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 09:00:24.941426 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:24.940968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55872)
0806 09:00:24.941186 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:24.941286 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 09:00:24.941705 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
I20260806 09:00:25.035975 9598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 4
I20260806 09:00:25.037222 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.036663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55886)
0806 09:00:25.036925 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.037037 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 09:00:25.038223 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.037847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55896)
0806 09:00:25.038005 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.038091 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 09:00:25.038472 9598 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
W20260806 09:00:25.043280 9607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:25.043767 9607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:25.043928 9607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 09:00:25.047932 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 3
I20260806 09:00:25.049067 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.048530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55900)
0806 09:00:25.048759 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.048868 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 09:00:25.050294 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.049876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55910)
0806 09:00:25.050066 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.050157 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 09:00:25.050562 9485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:37153: Network error: Client connection negotiation failed: client connection to 127.22.98.60:37153: connect: Connection refused (error 111)
W20260806 09:00:25.053620 9607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:25.053737 9607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:25.053814 9607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:25.053893 9607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:25.053952 9607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:25.064389 9607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:42539,127.22.98.61:34137,127.22.98.60:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:37153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.98.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:25.065919 9607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:25.067340 9607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:25.079876 9616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:25.079876 9621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:25.079923 9618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:25.081002 9607 server_base.cc:1034] running on GCE node
I20260806 09:00:25.081851 9607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:25.083320 9607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:25.084524 9607 hybrid_clock.cc:648] HybridClock initialized: now 1786006825084444 us; error 76 us; skew 500 ppm
I20260806 09:00:25.084923 9607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006825, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:25.088207 9607 init.cc:377] Logged in from keytab as kudu/127.22.98.60@KRBTEST.COM (short username kudu)
I20260806 09:00:25.089849 9607 webserver.cc:460] Webserver started at http://127.22.98.60:36719/ using document root <none> and password file <none>
I20260806 09:00:25.090571 9607 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:25.090695 9607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:25.091012 9607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:25.094229 9607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data/instance:
uuid: "2fee845c93614ce6a6234e8488dd7b2e"
format_stamp: "Formatted at 2026-08-06 09:00:25 on dist-test-slave-jpj1"
server_key: "d3e430cd4085ad6433d452ce1913168c"
server_key_iv: "6b57eefd62703f71c040d28a9230dfbc"
server_key_version: "encryptionkey@0"
I20260806 09:00:25.095118 9607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal/instance:
uuid: "2fee845c93614ce6a6234e8488dd7b2e"
format_stamp: "Formatted at 2026-08-06 09:00:25 on dist-test-slave-jpj1"
server_key: "d3e430cd4085ad6433d452ce1913168c"
server_key_iv: "6b57eefd62703f71c040d28a9230dfbc"
server_key_version: "encryptionkey@0"
I20260806 09:00:25.101039 9607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:00:25.104833 9629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:25.106369 9607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:00:25.106566 9607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "2fee845c93614ce6a6234e8488dd7b2e"
format_stamp: "Formatted at 2026-08-06 09:00:25 on dist-test-slave-jpj1"
server_key: "d3e430cd4085ad6433d452ce1913168c"
server_key_iv: "6b57eefd62703f71c040d28a9230dfbc"
server_key_version: "encryptionkey@0"
I20260806 09:00:25.106783 9607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:25.144982 9607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:25.149739 9607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:25.150022 9607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:25.168077 9607 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.60:37153
I20260806 09:00:25.168126 9698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.60:37153 every 8 connection(s)
I20260806 09:00:25.169485 9607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
I20260806 09:00:25.173250 9699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:25.177086 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9607
I20260806 09:00:25.177309 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786006610327111-22920-0/minicluster-data/master-2/wal/instance
I20260806 09:00:25.177809 22920 external_mini_cluster.cc:1428] Setting key f9ce1ae76aaf874e19fe78e433393ca6
I20260806 09:00:25.180960 9699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006825, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:00:25.201964 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.185362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.22.98.62:42539)
0806 09:00:25.185905 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.185912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.186573 (+ 661us) server_negotiation.cc:407] Connection header received
0806 09:00:25.186707 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.186711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.186780 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.186874 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.188197 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.188918 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.190326 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.190758 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.190871 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.194327 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.194333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.194336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.194373 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.197681 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.198845 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.198851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.198855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.199021 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.199583 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.199588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.199591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.199979 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.200125 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.201284 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.201523 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":327,"threads_started":1}
I20260806 09:00:25.203167 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.185340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:42539 (local address 127.0.0.1:46854)
0806 09:00:25.186321 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.186361 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:25.186571 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:25.186978 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:25.186986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:25.187726 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:25.188029 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.188039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.189069 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.189073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.190185 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.190198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.190996 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.191003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.191131 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.191718 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:25.191740 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:25.194095 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:25.197855 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.197869 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.197883 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.198687 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.199189 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.199194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.199197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.199429 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.200131 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:25.200166 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:25.200853 (+ 687us) client_negotiation.cc:769] Sending connection context
0806 09:00:25.201148 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 09:00:25.201656 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":564,"threads_started":1}
I20260806 09:00:25.203186 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.179893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.22.98.60:37153)
0806 09:00:25.180699 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.180716 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.180769 (+ 53us) server_negotiation.cc:407] Connection header received
0806 09:00:25.182946 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.182983 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.183564 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.184049 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.185919 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.187734 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.189148 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.189967 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.190150 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.194876 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.194889 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.194909 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.194954 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.198538 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.199573 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.199581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.199585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.199749 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.200190 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.200195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.200198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.200727 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.201011 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.201411 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.201767 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":572,"thread_start_us":348,"threads_started":1}
I20260806 09:00:25.205102 9699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006825, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:25.218910 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.205871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34137 (local address 127.0.0.1:53478)
0806 09:00:25.206188 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.206213 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:25.206329 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:25.206849 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:25.206854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:25.207223 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:25.207356 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.207364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.208357 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.208363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.209227 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.209239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.209967 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.209972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.210179 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.210808 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:25.210831 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:25.213211 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:25.216335 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.216341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.216345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.216886 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.217265 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.217270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.217274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.217455 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.217961 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:25.217965 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:25.218106 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 09:00:25.218242 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 09:00:25.218538 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260806 09:00:25.218932 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.205874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.22.98.61:34137)
0806 09:00:25.206082 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.206086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.206294 (+ 208us) server_negotiation.cc:407] Connection header received
0806 09:00:25.206482 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.206488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.206570 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.206641 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.207507 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.208218 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.209361 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.209832 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.209940 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.213433 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.213440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.213445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.213490 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.216198 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.217019 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.217023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.217026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.217158 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.217549 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.217552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.217554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.217838 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.217981 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.218360 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.218559 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:00:25.219947 9699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:00:25.220436 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.206330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.22.98.62:42539)
0806 09:00:25.206487 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.206492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.206507 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:00:25.206618 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.206623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.206686 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.206789 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.208039 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.208723 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.209919 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.210425 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.210540 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.214905 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.214910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.214914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.214947 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.217819 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.218422 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.218428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.218431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.218544 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.218907 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.218913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.218916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.219258 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.219514 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.219913 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.220127 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006825, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:00:25.227766 9485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 4
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:00:25.234647 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.220738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55914)
0806 09:00:25.220912 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.220929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:25.221020 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:25.221306 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:25.221310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:25.221736 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:25.221911 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.221924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.222982 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.222987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.223915 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.223928 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.224607 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.224613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.224841 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.225388 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:25.225406 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:25.227724 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:25.231680 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.231685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.231689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.232172 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.232535 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.232540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.232543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.232714 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.233491 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:25.233496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:25.233652 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 09:00:25.233824 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 09:00:25.234196 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 09:00:25.234745 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.220932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.22.98.60:37153)
0806 09:00:25.221074 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.221080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.221095 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:00:25.221134 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.221139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.221192 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.221272 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.222068 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.222863 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.224040 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.224478 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.224583 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.227952 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.227960 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.227964 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.228000 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.231515 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.232284 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.232290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.232294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.232432 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.232845 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.232850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.232855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.233281 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.233442 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.233968 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.234396 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":46}
I20260806 09:00:25.237978 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.222392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.22.98.61:34137)
0806 09:00:25.222564 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.222570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.222586 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:00:25.222634 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.222637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.222688 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.222766 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.224031 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.224733 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.226022 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.226500 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.226633 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.231430 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.231436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.231439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.231474 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.235069 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.235984 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.235990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.235994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.236144 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.236552 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.236556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.236558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.236845 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.237135 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.237414 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.237640 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:00:25.243990 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:00:25.249516 9699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: Bootstrap starting.
I20260806 09:00:25.254401 9699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:25.255105 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.228420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55928)
0806 09:00:25.228624 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.228645 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:25.228773 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:25.235600 (+ 6827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:25.235608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:25.236164 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:25.236399 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.236412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.238021 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.238029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.239600 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.239621 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.240402 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.240408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.240589 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.241181 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:25.241200 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:25.243835 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:25.251783 (+ 7948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.251791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.251796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.252416 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.252932 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.252938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.252942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.253123 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.253958 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:25.253966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:25.254168 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 09:00:25.254304 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 09:00:25.254661 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 09:00:25.256093 9699 log.cc:824] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:25.256675 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.228927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.22.98.60:37153)
0806 09:00:25.235215 (+ 6288us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.235220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.235237 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:00:25.235282 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.235286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.235358 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.235459 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.236585 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.237807 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.239743 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.240248 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.240363 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.247726 (+ 7363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.247734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.247739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.247784 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.251513 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.252602 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.252608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.252612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.252795 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.253271 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.253276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.253281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.253801 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.255779 (+ 1978us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.255962 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.256226 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6181,"thread_start_us":15093,"threads_started":1}
I20260806 09:00:25.263607 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: Bootstrap starting.
I20260806 09:00:25.266024 9699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: No bootstrap required, opened a new log
I20260806 09:00:25.269193 9485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:25.270893 9485 log.cc:824] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:25.277338 9699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.277985 9699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:25.278180 9699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fee845c93614ce6a6234e8488dd7b2e, State: Initialized, Role: FOLLOWER
I20260806 09:00:25.278344 9485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: No bootstrap required, opened a new log
I20260806 09:00:25.279072 9699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.282823 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.249603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.22.98.61:34137)
0806 09:00:25.249771 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.249777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.256710 (+ 6933us) server_negotiation.cc:407] Connection header received
0806 09:00:25.257285 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.257303 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.257396 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.257471 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.260157 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.261399 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.263673 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.264333 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.264444 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.271758 (+ 7314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.271766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.271771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.271822 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.276378 (+ 4556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.278022 (+ 1644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.278028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.278031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.278218 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.278767 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.278773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.278776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.279174 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.279436 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.282139 (+ 2703us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.282433 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 09:00:25.286233 9721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.288832 9721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.288695 9699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:25.289618 9485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.290146 9485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:25.290352 9485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a614776d6b14ba6b75c5cceb06991b6, State: Initialized, Role: FOLLOWER
I20260806 09:00:25.291457 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.257497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.22.98.60:37153)
0806 09:00:25.257621 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.257626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.273605 (+ 15979us) server_negotiation.cc:407] Connection header received
0806 09:00:25.273792 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.273799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.273938 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.274057 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.276045 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.277125 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.279196 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.279878 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.279982 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.283180 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.283188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.283193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.283239 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.288058 (+ 4819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.288874 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.288880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.288883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.289032 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.289533 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.289538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.289541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.289958 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.290204 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.290847 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.291090 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 09:00:25.291977 9485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.292912 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.293319 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.295434 9485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:25.295491 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.249144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46866 (local address 127.22.98.62:42539)
0806 09:00:25.249348 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.249354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.257355 (+ 8001us) server_negotiation.cc:407] Connection header received
0806 09:00:25.257579 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.257585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.257674 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.257742 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.268870 (+ 11128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.270072 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.279994 (+ 9922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.280828 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.280936 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.287404 (+ 6468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.287411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.287415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.287458 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.291132 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.293230 (+ 2098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.293236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.293239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.293407 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.293827 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.293832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.293835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.294210 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.294470 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.294856 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.295107 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 09:00:25.307771 9757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:00:25.308045 9757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 09:00:25.318197 9760 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:00:25.318471 9760 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:00:25.308260 9607 master_runner.cc:186] Error getting master registration for 127.22.98.61:34137: OK, instance_id { permanent_uuid: "4d90e2041dfd4083b55e841612435616" instance_seqno: 1786006824606132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:00:25.319638 9607 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:00:25.316273 9389 master_runner.cc:186] Error getting master registration for 127.22.98.61:34137: OK, instance_id { permanent_uuid: "4d90e2041dfd4083b55e841612435616" instance_seqno: 1786006824606132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:00:25.325739 9389 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 09:00:25.326380 9419 tablet.cc:2369] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:25.337692 9598 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } attempt: 5
Aug 06 09:00:25 dist-test-slave-jpj1 krb5kdc[9370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006824, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:00:25.357093 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.338441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:37153 (local address 127.0.0.1:55936)
0806 09:00:25.338899 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:25.338923 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:25.339065 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:25.339802 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:25.339808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:25.340389 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:25.340611 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.340626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.342124 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.342130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.343621 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:25.343643 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.344796 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.344804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:25.345032 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.345988 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:25.346072 (+ 84us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:25.349406 (+ 3334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:25.353932 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.353938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.353943 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.354565 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.355071 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:25.355077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:25.355081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:25.355266 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:25.355981 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:25.355987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:25.356162 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 09:00:25.356347 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 09:00:25.356677 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315}
I20260806 09:00:25.357132 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:25.339237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.22.98.60:37153)
0806 09:00:25.339441 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:25.339447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:25.339466 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:00:25.339526 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:25.339557 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:25.339635 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:25.339752 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:25.340790 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.341909 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.343808 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:25.344587 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:25.344739 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:25.349715 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:25.349723 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:25.349728 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:25.349773 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:25.353737 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.354725 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.354732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.354736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.354939 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:25.355405 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:25.355411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:25.355415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:25.355847 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:25.356071 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:25.356479 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 09:00:25.356739 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:00:25.363914 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: Bootstrap starting.
I20260806 09:00:25.368459 9598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:25.370007 9598 log.cc:824] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:25.373646 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: No bootstrap required, opened a new log
I20260806 09:00:25.384938 9598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.385452 9598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:25.385579 9598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d90e2041dfd4083b55e841612435616, State: Initialized, Role: FOLLOWER
I20260806 09:00:25.385743 9721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:25.386096 9721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.386469 9598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.387881 9721 leader_election.cc:288] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a614776d6b14ba6b75c5cceb06991b6 (127.22.98.62:42539), 4d90e2041dfd4083b55e841612435616 (127.22.98.61:34137)
I20260806 09:00:25.389190 9451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fee845c93614ce6a6234e8488dd7b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a614776d6b14ba6b75c5cceb06991b6" is_pre_election: true
I20260806 09:00:25.389863 9451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fee845c93614ce6a6234e8488dd7b2e in term 0.
I20260806 09:00:25.391453 9634 leader_election.cc:302] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a614776d6b14ba6b75c5cceb06991b6, 2fee845c93614ce6a6234e8488dd7b2e; no voters:
I20260806 09:00:25.392056 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:00:25.392226 9721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:00:25.392333 9721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:25.393972 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.394363 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.396332 9598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:25.396466 9559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fee845c93614ce6a6234e8488dd7b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d90e2041dfd4083b55e841612435616" is_pre_election: true
I20260806 09:00:25.397091 9559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fee845c93614ce6a6234e8488dd7b2e in term 0.
I20260806 09:00:25.401126 9721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
W20260806 09:00:25.403210 9636 tablet.cc:2369] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:25.405215 9451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fee845c93614ce6a6234e8488dd7b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a614776d6b14ba6b75c5cceb06991b6"
I20260806 09:00:25.405463 9451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:25.411751 9451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fee845c93614ce6a6234e8488dd7b2e in term 1.
I20260806 09:00:25.412334 9634 leader_election.cc:302] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a614776d6b14ba6b75c5cceb06991b6, 2fee845c93614ce6a6234e8488dd7b2e; no voters:
I20260806 09:00:25.415805 9559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fee845c93614ce6a6234e8488dd7b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d90e2041dfd4083b55e841612435616"
I20260806 09:00:25.416052 9559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:25.419494 9559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fee845c93614ce6a6234e8488dd7b2e in term 1.
I20260806 09:00:25.424463 9721 leader_election.cc:288] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [CANDIDATE]: Term 1 election: Requested vote from peers 1a614776d6b14ba6b75c5cceb06991b6 (127.22.98.62:42539), 4d90e2041dfd4083b55e841612435616 (127.22.98.61:34137)
I20260806 09:00:25.424957 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:25.425450 9721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [term 1 LEADER]: Becoming Leader. State: Replica: 2fee845c93614ce6a6234e8488dd7b2e, State: Running, Role: LEADER
I20260806 09:00:25.426385 9721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } }
I20260806 09:00:25.442629 9770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fee845c93614ce6a6234e8488dd7b2e. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.443199 9770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:25.446236 9773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:25.448748 9773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:25.469254 9559 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [term 1 FOLLOWER]: Refusing update from remote peer 2fee845c93614ce6a6234e8488dd7b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:25.470578 9770 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:25.482349 9451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [term 1 FOLLOWER]: Refusing update from remote peer 2fee845c93614ce6a6234e8488dd7b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:25.483453 9770 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:25.513134 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fee845c93614ce6a6234e8488dd7b2e. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.513510 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.521261 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fee845c93614ce6a6234e8488dd7b2e. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.521631 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.523180 9777 mvcc.cc:205] Tried to move back new op lower bound from 7315483957107404800 to 7315483956962279424. Current Snapshot: MvccSnapshot[applied={T|T < 7315483957107404800}]
I20260806 09:00:25.538535 9783 mvcc.cc:205] Tried to move back new op lower bound from 7315483957107404800 to 7315483956962279424. Current Snapshot: MvccSnapshot[applied={T|T < 7315483957107404800}]
I20260806 09:00:25.540234 9770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.540555 9770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:25.540855 9770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.541070 9770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:25.546367 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.546682 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.551013 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fee845c93614ce6a6234e8488dd7b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a614776d6b14ba6b75c5cceb06991b6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 42539 } } peers { permanent_uuid: "4d90e2041dfd4083b55e841612435616" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34137 } } peers { permanent_uuid: "2fee845c93614ce6a6234e8488dd7b2e" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 37153 } } }
I20260806 09:00:25.551280 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:00:25.553356 9773 catalog_manager.cc:1347] Generated new cluster ID: 0292ec07d2754acdacafbd409389dcf3
I20260806 09:00:25.553478 9773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260806 09:00:25.559628 9789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:00:25.559878 9789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:00:25.588454 9773 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:00:25.590677 9773 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:25.611508 9773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fee845c93614ce6a6234e8488dd7b2e: Generated new TSK 0
I20260806 09:00:25.612679 9773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:00:26.310277 9757 catalog_manager.cc:1259] Loaded cluster ID: 0292ec07d2754acdacafbd409389dcf3
I20260806 09:00:26.310396 9757 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: loading cluster ID for follower catalog manager: success
I20260806 09:00:26.312960 9757 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: acquiring CA information for follower catalog manager: success
I20260806 09:00:26.314810 9757 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a614776d6b14ba6b75c5cceb06991b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:00:26.561758 9789 catalog_manager.cc:1259] Loaded cluster ID: 0292ec07d2754acdacafbd409389dcf3
I20260806 09:00:26.561868 9789 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: loading cluster ID for follower catalog manager: success
I20260806 09:00:26.564404 9789 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: acquiring CA information for follower catalog manager: success
I20260806 09:00:26.566213 9789 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d90e2041dfd4083b55e841612435616: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:00:26.745740 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9389
I20260806 09:00:26.747268 9477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:26.856987 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9389
I20260806 09:00:26.885860 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9495
I20260806 09:00:26.886987 9586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:26.992710 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9495
I20260806 09:00:27.018450 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9607
I20260806 09:00:27.019603 9693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:27.129151 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9607
2026-08-06T09:00:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5025 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:00:27 dist-test-slave-jpj1 krb5kdc[9813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:00:27 dist-test-slave-jpj1 krb5kdc[9813](info): set up 2 sockets
Aug 06 09:00:27 dist-test-slave-jpj1 krb5kdc[9813](info): commencing operation
krb5kdc: starting...
W20260806 09:00:28.331979 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:00:28 dist-test-slave-jpj1 krb5kdc[9813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:00:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:00:28.615926 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41501
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:38919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:28.977882 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:28.978204 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:28.978324 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:28.985576 9834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:28.985707 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:28.985759 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:28.985839 9834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:28.985899 9834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:28.996173 9834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:38919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:41501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:28.997556 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:28.998905 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:29.010515 9840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:29.010697 9844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:29.010643 9842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:29.011615 9834 server_base.cc:1034] running on GCE node
I20260806 09:00:29.012434 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:29.013898 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:29.015121 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786006829015070 us; error 35 us; skew 500 ppm
I20260806 09:00:29.015493 9834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:29 dist-test-slave-jpj1 krb5kdc[9813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006829, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:29.022620 9834 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:29.024410 9834 webserver.cc:460] Webserver started at http://127.22.98.62:43807/ using document root <none> and password file <none>
I20260806 09:00:29.025174 9834 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:29.025295 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:29.025629 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:29.029114 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "13522560380c4dcaab6a42bbec91d5b3"
format_stamp: "Formatted at 2026-08-06 09:00:29 on dist-test-slave-jpj1"
server_key: "86514213397e5fb93e364b27efc6bc48"
server_key_iv: "8e7bd73c5efaa89f43c9fe199a345325"
server_key_version: "encryptionkey@0"
I20260806 09:00:29.029966 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "13522560380c4dcaab6a42bbec91d5b3"
format_stamp: "Formatted at 2026-08-06 09:00:29 on dist-test-slave-jpj1"
server_key: "86514213397e5fb93e364b27efc6bc48"
server_key_iv: "8e7bd73c5efaa89f43c9fe199a345325"
server_key_version: "encryptionkey@0"
I20260806 09:00:29.035741 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:00:29.039565 9852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:29.040963 9834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:29.041142 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "13522560380c4dcaab6a42bbec91d5b3"
format_stamp: "Formatted at 2026-08-06 09:00:29 on dist-test-slave-jpj1"
server_key: "86514213397e5fb93e364b27efc6bc48"
server_key_iv: "8e7bd73c5efaa89f43c9fe199a345325"
server_key_version: "encryptionkey@0"
I20260806 09:00:29.041345 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:29.059579 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:29.063656 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:29.063933 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:29.083045 9834 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:41501
I20260806 09:00:29.083067 9925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:41501 every 8 connection(s)
I20260806 09:00:29.084558 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:29.088419 9926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:29.094094 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9834
I20260806 09:00:29.094333 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:00:29.095216 22920 external_mini_cluster.cc:1428] Setting key ac7b683913547593141c610dc5ec9662
I20260806 09:00:29.100244 9926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: Bootstrap starting.
I20260806 09:00:29.104128 9926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:29.105453 9926 log.cc:824] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:29.109090 9926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: No bootstrap required, opened a new log
Aug 06 09:00:29 dist-test-slave-jpj1 krb5kdc[9813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:29.117738 9926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } }
I20260806 09:00:29.118139 9926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:29.118263 9926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13522560380c4dcaab6a42bbec91d5b3, State: Initialized, Role: FOLLOWER
I20260806 09:00:29.119028 9926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } }
I20260806 09:00:29.119383 9926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:29.119522 9926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:29.119738 9926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:29.120653 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:29.097409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.22.98.62:41501)
0806 09:00:29.098317 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:29.098330 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:29.098373 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:00:29.100250 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:29.100277 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:29.100708 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:29.101043 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:29.103010 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:29.104590 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:29.106463 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:29.107404 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:29.107608 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:29.112691 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:29.112707 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:29.112730 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:29.112776 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:29.116069 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:29.116846 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:29.116854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:29.116862 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:29.117052 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:29.117470 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:29.117474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:29.117476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:29.118077 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:29.118386 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:29.118959 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 09:00:29.119417 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":427,"threads_started":1}
I20260806 09:00:29.122218 9926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } }
I20260806 09:00:29.122777 9926 leader_election.cc:302] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13522560380c4dcaab6a42bbec91d5b3; no voters:
I20260806 09:00:29.123868 9926 leader_election.cc:288] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:00:29.124027 9932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:29.124997 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:00:29.125844 9926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:29.129711 9932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [term 1 LEADER]: Becoming Leader. State: Replica: 13522560380c4dcaab6a42bbec91d5b3, State: Running, Role: LEADER
I20260806 09:00:29.130424 9932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } }
I20260806 09:00:29.133831 9933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13522560380c4dcaab6a42bbec91d5b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } } }
I20260806 09:00:29.133795 9936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13522560380c4dcaab6a42bbec91d5b3. Latest consensus state: current_term: 1 leader_uuid: "13522560380c4dcaab6a42bbec91d5b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13522560380c4dcaab6a42bbec91d5b3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41501 } } }
I20260806 09:00:29.134279 9933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:29.134277 9936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3 [sys.catalog]: This master's current role is: LEADER
W20260806 09:00:29.139814 9954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:00:29.139928 9954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:00:29.140040 9956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:29.142182 9956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:29.151620 9956 catalog_manager.cc:1347] Generated new cluster ID: 2dd414bfdd42409abe547aac3c638eda
I20260806 09:00:29.151758 9956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:29.178328 9956 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:00:29.179893 9956 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:29.187577 9956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13522560380c4dcaab6a42bbec91d5b3: Generated new TSK 0
I20260806 09:00:29.188665 9956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:00:29.239769 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9834
I20260806 09:00:29.241068 9920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:29.339447 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9834
2026-08-06T09:00:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2205 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:00:29 dist-test-slave-jpj1 krb5kdc[9970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:00:29 dist-test-slave-jpj1 krb5kdc[9970](info): set up 2 sockets
Aug 06 09:00:29 dist-test-slave-jpj1 krb5kdc[9970](info): commencing operation
krb5kdc: starting...
W20260806 09:00:30.538014 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:00:30 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:00:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:00:30.825273 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:31.178040 9986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:31.178390 9986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:31.178525 9986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:31.186265 9986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:31.186396 9986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:31.186444 9986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:31.186507 9986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:31.186570 9986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:31.197271 9986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:31.198688 9986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:31.200135 9986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:31.212399 9995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:31.212419 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:31.212446 9994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:31.217178 9986 server_base.cc:1034] running on GCE node
I20260806 09:00:31.218441 9986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:31.220615 9986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:31.222012 9986 hybrid_clock.cc:648] HybridClock initialized: now 1786006831221943 us; error 61 us; skew 500 ppm
I20260806 09:00:31.222496 9986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:31 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:31.227401 9986 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:31.229271 9986 webserver.cc:460] Webserver started at http://127.22.98.62:41591/ using document root <none> and password file <none>
I20260806 09:00:31.230089 9986 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:31.230221 9986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:31.230502 9986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:31.233902 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "0e8031c1a8d8486ca3427a55a011c8da"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "4e0c48bf16ab4ebfc4a397c890667412"
server_key_iv: "f7a4b4f068f5eb1c38cf16e38b330b9e"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.234728 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "0e8031c1a8d8486ca3427a55a011c8da"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "4e0c48bf16ab4ebfc4a397c890667412"
server_key_iv: "f7a4b4f068f5eb1c38cf16e38b330b9e"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.240461 9986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:00:31.244320 10010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:31.246075 9986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:00:31.246300 9986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "0e8031c1a8d8486ca3427a55a011c8da"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "4e0c48bf16ab4ebfc4a397c890667412"
server_key_iv: "f7a4b4f068f5eb1c38cf16e38b330b9e"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.246526 9986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:31.271030 9986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:31.275945 9986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:31.276237 9986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:31.295888 9986 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:43991
I20260806 09:00:31.295943 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:43991 every 8 connection(s)
I20260806 09:00:31.297346 9986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:31.301391 10078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:31.304752 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 9986
I20260806 09:00:31.304949 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:00:31.305616 22920 external_mini_cluster.cc:1428] Setting key 642662953c816495ee89bde2ba4c5e38
I20260806 09:00:31.315357 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap starting.
I20260806 09:00:31.319139 10078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:31.320276 10078 log.cc:824] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:00:31 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:31.323294 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: No bootstrap required, opened a new log
I20260806 09:00:31.330497 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:31.307900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.22.98.62:43991)
0806 09:00:31.308859 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:31.308875 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:31.308928 (+ 53us) server_negotiation.cc:407] Connection header received
0806 09:00:31.310892 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:31.310926 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:31.311513 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:31.312049 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:31.313439 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.315469 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.317011 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.318066 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.318264 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:31.322726 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:31.322739 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:31.322757 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:31.322801 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:31.325947 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:31.326823 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:31.326831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:31.326838 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:31.327017 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:31.327429 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:31.327433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:31.327438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:31.328075 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:31.328409 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:31.328877 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 09:00:31.329327 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":445,"threads_started":1}
I20260806 09:00:31.331444 10078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:31.331878 10078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:31.332000 10078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Initialized, Role: FOLLOWER
I20260806 09:00:31.332665 10078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:31.332983 10078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:31.333130 10078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:31.333323 10078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:31.335911 10078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:31.336486 10078 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e8031c1a8d8486ca3427a55a011c8da; no voters:
I20260806 09:00:31.337061 10078 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:00:31.337224 10084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:31.337885 10084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 LEADER]: Becoming Leader. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Running, Role: LEADER
I20260806 09:00:31.338419 10084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:31.339124 10078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:31.341789 10086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8031c1a8d8486ca3427a55a011c8da. Latest consensus state: current_term: 1 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:31.342178 10086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:31.342927 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:31.343885 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:31.344432 10093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:31.350505 10093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:31.360570 10093 catalog_manager.cc:1347] Generated new cluster ID: 8f0c54db65824e94b4e705b2a99d3ef8
I20260806 09:00:31.360689 10093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:31.378350 10093 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:00:31.379707 10093 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:31.388738 10093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Generated new TSK 0
I20260806 09:00:31.389950 10093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:00:31.463758 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:31.817212 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:31.817516 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:31.817623 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:31.825044 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:31.825165 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:31.825289 10115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:31.837808 10115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:31.839138 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:31.840586 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:31.854019 10123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:31.853972 10125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:31.853935 10128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:31.860350 10115 server_base.cc:1034] running on GCE node
I20260806 09:00:31.861047 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:31.862073 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:31.863276 10115 hybrid_clock.cc:648] HybridClock initialized: now 1786006831863214 us; error 58 us; skew 500 ppm
I20260806 09:00:31.863627 10115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:31 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:31.866806 10115 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:31.868422 10115 webserver.cc:460] Webserver started at http://127.22.98.1:42289/ using document root <none> and password file <none>
I20260806 09:00:31.869164 10115 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:31.869294 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:31.869603 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:31.872699 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "3f82ee45bc2f4cd197b5baee374e824f"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "a95ec2051d9c55d4d77223963b78c437"
server_key_iv: "ce111c996712664ab36ad19fbb2b8352"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.873539 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "3f82ee45bc2f4cd197b5baee374e824f"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "a95ec2051d9c55d4d77223963b78c437"
server_key_iv: "ce111c996712664ab36ad19fbb2b8352"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.879274 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:00:31.883105 10141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:31.884737 10115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:31.884903 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "3f82ee45bc2f4cd197b5baee374e824f"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "a95ec2051d9c55d4d77223963b78c437"
server_key_iv: "ce111c996712664ab36ad19fbb2b8352"
server_key_version: "encryptionkey@0"
I20260806 09:00:31.885082 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:31.901366 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:31.905526 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:31.905783 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:31.906735 10115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:31.908111 10115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:31.908190 10115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:31.908291 10115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:31.908337 10115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:31.939304 10115 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:35047
I20260806 09:00:31.939342 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:35047 every 8 connection(s)
I20260806 09:00:31.940797 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:31.943575 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 10115
I20260806 09:00:31.943819 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:00:31.944715 22920 external_mini_cluster.cc:1428] Setting key 8374e82f37b67ffefd5809bc1152ee1d
Aug 06 09:00:31 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006831, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:31.962329 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:31.943430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:35949 (local address 127.22.98.62:43991)
0806 09:00:31.943946 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:31.943951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:31.945292 (+ 1341us) server_negotiation.cc:407] Connection header received
0806 09:00:31.947670 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:31.947676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:31.947760 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:31.947856 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:31.949496 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.950235 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.952068 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.952486 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.952610 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:31.956481 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:31.956487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:31.956491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:31.956525 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:31.959082 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:31.959923 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:31.959928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:31.959930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:31.960058 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:31.960528 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:31.960533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:31.960535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:31.960820 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:31.960955 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:31.961733 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 09:00:31.961980 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":266,"threads_started":1}
I20260806 09:00:31.963356 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:31.944141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:35949)
0806 09:00:31.945097 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:31.945173 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:31.947366 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:31.948127 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:31.948135 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:31.948707 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:31.949321 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:31.949341 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.950390 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.950396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:31.951872 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:31.951886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:31.952755 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:31.952763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:31.952999 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:31.953700 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:31.953726 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:31.956222 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:31.959252 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:31.959262 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:31.959278 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:31.959790 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:31.960176 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:31.960180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:31.960182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:31.960407 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:31.960950 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:31.960959 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:31.961325 (+ 366us) client_negotiation.cc:769] Sending connection context
0806 09:00:31.961577 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 09:00:31.962149 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":442,"threads_started":1}
I20260806 09:00:31.965085 10292 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:31.965485 10292 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:31.966424 10292 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:31.969084 10032 ts_manager.cc:194] Registered new tserver with Master: 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047)
I20260806 09:00:31.971125 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:35949
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 09:00:32.012036 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:32.370991 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:32.371281 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:32.371376 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:32.378963 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:32.379083 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:32.379200 10303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:32.389310 10303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:32.390635 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:32.392022 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:32.405514 10311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:32.405470 10313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:32.405434 10310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:32.411789 10303 server_base.cc:1034] running on GCE node
I20260806 09:00:32.412509 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:32.413589 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:32.414847 10303 hybrid_clock.cc:648] HybridClock initialized: now 1786006832414787 us; error 47 us; skew 500 ppm
I20260806 09:00:32.415215 10303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:32 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006832, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:32.418720 10303 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:32.420620 10303 webserver.cc:460] Webserver started at http://127.22.98.2:35843/ using document root <none> and password file <none>
I20260806 09:00:32.421391 10303 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:32.421530 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:32.421864 10303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:32.425164 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "44c0f1fbdd444f65b9160913c130bbfc"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "3fb4d5379a70cb48846e81f08106a4b0"
server_key_iv: "94ea620b1768e7266f6199ca0cd2842e"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.426079 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "44c0f1fbdd444f65b9160913c130bbfc"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "3fb4d5379a70cb48846e81f08106a4b0"
server_key_iv: "94ea620b1768e7266f6199ca0cd2842e"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.431927 10303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 09:00:32.435986 10324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:32.437628 10303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:00:32.437858 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "44c0f1fbdd444f65b9160913c130bbfc"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "3fb4d5379a70cb48846e81f08106a4b0"
server_key_iv: "94ea620b1768e7266f6199ca0cd2842e"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.438097 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:32.470366 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:32.475425 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:32.475785 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:32.476882 10303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:32.478482 10303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:32.478575 10303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:32.478757 10303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:32.478823 10303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:32.511754 10303 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46475
I20260806 09:00:32.511791 10475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46475 every 8 connection(s)
I20260806 09:00:32.513062 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:32.522516 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 10303
I20260806 09:00:32.522713 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:00:32.523197 22920 external_mini_cluster.cc:1428] Setting key 159eff1db05ae162ae44abdaab2c8e9a
Aug 06 09:00:32 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006832, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:32.534852 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:32.515738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:50637 (local address 127.22.98.62:43991)
0806 09:00:32.516248 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:32.516253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:32.517442 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 09:00:32.519497 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:32.519503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:32.519631 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:32.519743 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:32.521514 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:32.522271 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:32.524193 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:32.524724 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:32.524892 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:32.528541 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:32.528548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:32.528553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:32.528592 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:32.531680 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:32.532413 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:32.532418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:32.532421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:32.532579 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:32.533043 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:32.533047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:32.533050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:32.533377 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:32.533514 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:32.534260 (+ 746us) server_negotiation.cc:299] Negotiation successful
0806 09:00:32.534479 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":385,"thread_start_us":246,"threads_started":1}
I20260806 09:00:32.535737 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:32.516393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:50637)
0806 09:00:32.517237 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:32.517292 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:32.519127 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:32.520052 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:32.520062 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:32.520662 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:32.521315 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:32.521341 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:32.522434 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:32.522438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:32.524025 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:32.524043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:32.524982 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:32.524990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:32.525264 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:32.526012 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:32.526036 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:32.528294 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:32.531825 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:32.531833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:32.531847 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:32.532281 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:32.532683 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:32.532687 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:32.532689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:32.532924 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:32.533502 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:32.533508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:32.533887 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 09:00:32.534124 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 09:00:32.534534 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":350,"threads_started":1}
I20260806 09:00:32.537246 10476 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:32.537545 10476 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:32.538293 10476 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:32.540073 10032 ts_manager.cc:194] Registered new tserver with Master: 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475)
I20260806 09:00:32.540927 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:50637
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:00:32.589103 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:32.933687 10486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:32.934011 10486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:32.934158 10486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:32.943135 10486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:32.943284 10486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:32.943459 10486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:00:32.954433 10486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:32.955806 10486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:32.957218 10486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:32.970999 10492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:32.970984 10491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:32.970983 10494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:32.971843 10486 server_base.cc:1034] running on GCE node
I20260806 09:00:32.972496 10486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:32.973474 10486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:32.974681 10486 hybrid_clock.cc:648] HybridClock initialized: now 1786006832974642 us; error 60 us; skew 500 ppm
I20260806 09:00:32.975030 10486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:00:32.975098 10292 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
Aug 06 09:00:32 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006832, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:32.978502 10486 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:32.980041 10486 webserver.cc:460] Webserver started at http://127.22.98.3:43137/ using document root <none> and password file <none>
I20260806 09:00:32.980749 10486 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:32.980881 10486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:32.981182 10486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:32.984447 10486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "0082fc114cce70f5871e9e44736ab002"
server_key_iv: "14e7425e86bb885bb1c572fa1fa804a4"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.985296 10486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "0082fc114cce70f5871e9e44736ab002"
server_key_iv: "14e7425e86bb885bb1c572fa1fa804a4"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.991266 10486 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 09:00:32.995080 10505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:32.996642 10486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:32.996816 10486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "0082fc114cce70f5871e9e44736ab002"
server_key_iv: "14e7425e86bb885bb1c572fa1fa804a4"
server_key_version: "encryptionkey@0"
I20260806 09:00:32.997028 10486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:33.009828 10486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:33.014119 10486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:33.014402 10486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:33.015426 10486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:33.016809 10486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:33.016892 10486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:33.017016 10486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:33.017071 10486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:33.051358 10486 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:45893
I20260806 09:00:33.051427 10662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:45893 every 8 connection(s)
I20260806 09:00:33.052793 10486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:33.058415 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 10486
I20260806 09:00:33.058583 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:00:33.059020 22920 external_mini_cluster.cc:1428] Setting key 2aa8d63b66e45adfad34b46e59409a28
Aug 06 09:00:33 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006832, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:33.073450 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.055455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:33411 (local address 127.22.98.62:43991)
0806 09:00:33.055948 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.055952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.057044 (+ 1092us) server_negotiation.cc:407] Connection header received
0806 09:00:33.059156 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.059162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.059260 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.059360 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:33.061134 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.061944 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.063512 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.063944 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.064063 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.067622 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:33.067627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:33.067630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:33.067667 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:33.070300 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.071052 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.071057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.071059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.071188 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.071631 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.071636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.071639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.071922 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:33.072067 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.072786 (+ 719us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.073020 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260806 09:00:33.074728 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.056098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:33411)
0806 09:00:33.056863 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.056913 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.058875 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.059663 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.059671 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.060344 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:33.060959 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.060979 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.062099 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.062103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.063362 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.063380 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.064195 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.064202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.064425 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.065083 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:33.065107 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:33.067370 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:33.070446 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:33.070456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:33.070474 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:33.070929 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:33.071307 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:33.071312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:33.071316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:33.071511 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:33.072054 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:33.072063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:33.072409 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.072658 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.073232 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":286,"threads_started":1}
I20260806 09:00:33.076144 10664 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:33.076493 10664 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:33.077333 10664 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:33.079110 10032 ts_manager.cc:194] Registered new tserver with Master: b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
I20260806 09:00:33.080025 10032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:33411
I20260806 09:00:33.087447 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:00:33.106276 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.092419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.22.98.62:43991)
0806 09:00:33.092661 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.092669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.092936 (+ 267us) server_negotiation.cc:407] Connection header received
0806 09:00:33.093103 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.093107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.093179 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.093262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:33.094786 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.095520 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.097067 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.097563 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.097672 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.100429 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:33.100436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:33.100439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:33.100477 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:33.103271 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.104135 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.104140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.104143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.104292 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.104714 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.104718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.104720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.105080 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:33.105280 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.105681 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.105935 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:00:33.111297 10032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:00:33.118791 10032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:00:33.146142 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.133121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.22.98.2:46475)
0806 09:00:33.133749 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.133759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.134496 (+ 737us) server_negotiation.cc:407] Connection header received
0806 09:00:33.134970 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.134980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.135281 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.135456 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:33.136573 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.138414 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.141727 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.143449 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.143599 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.143861 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.145547 (+ 1686us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.145769 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":354,"threads_started":1}
I20260806 09:00:33.147078 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.132997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46475 (local address 127.0.0.1:36498)
0806 09:00:33.134340 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.134394 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.134681 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.135853 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.135862 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.135897 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:33.136344 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.136357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.138652 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.138658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.141472 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.141495 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.144112 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.144120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.144379 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.144440 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.146386 (+ 1946us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.146655 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":11760,"threads_started":1}
I20260806 09:00:33.151619 10392 tablet_service.cc:1467] Processing CreateTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 (DEFAULT_TABLE table=test-table [id=8776aaa1df9d44728f26664e2292a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:33.153403 10392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50da3063069f49ddbeff3677cedd1ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:33.155947 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.134476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45918)
0806 09:00:33.147248 (+ 12772us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.147267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.147408 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.148086 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.148090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.148115 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:33.149494 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.149514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.151433 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.151438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.153617 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.153629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.154972 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.154978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.155147 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.155181 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.155297 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.155466 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12643,"mutex_wait_us":16,"thread_start_us":12461,"threads_started":1}
I20260806 09:00:33.156174 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.133727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.22.98.3:45893)
0806 09:00:33.148450 (+ 14723us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.148459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.148491 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:00:33.148572 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.148577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.148795 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.148988 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:33.149879 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.151007 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.153674 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.155100 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.155249 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.155468 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.155668 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.155847 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14554,"thread_start_us":13598,"threads_started":1}
I20260806 09:00:33.156098 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.133264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45893 (local address 127.0.0.1:51972)
0806 09:00:33.145149 (+ 11885us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.145172 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.145325 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.148972 (+ 3647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.148977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.149003 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:33.149657 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.149672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.151147 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.151153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.153509 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.153526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.155304 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.155310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.155480 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.155509 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.155623 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.155755 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11712,"mutex_wait_us":192,"thread_start_us":758,"threads_started":1}
I20260806 09:00:33.157032 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.135153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.22.98.1:35047)
0806 09:00:33.146755 (+ 11602us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.146766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.147573 (+ 807us) server_negotiation.cc:407] Connection header received
0806 09:00:33.147687 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.147693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.147896 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.148084 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:33.149750 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.151306 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.153766 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.154770 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.154893 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.155139 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.156231 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.156409 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11428,"thread_start_us":11310,"threads_started":1}
I20260806 09:00:33.160643 10209 tablet_service.cc:1467] Processing CreateTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 (DEFAULT_TABLE table=test-table [id=8776aaa1df9d44728f26664e2292a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:33.162163 10209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50da3063069f49ddbeff3677cedd1ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:33.165580 10576 tablet_service.cc:1467] Processing CreateTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 (DEFAULT_TABLE table=test-table [id=8776aaa1df9d44728f26664e2292a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:33.167295 10576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50da3063069f49ddbeff3677cedd1ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:33.169833 10693 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Bootstrap starting.
I20260806 09:00:33.173843 10693 tablet_bootstrap.cc:654] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:33.175266 10693 log.cc:824] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:33.178947 10694 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Bootstrap starting.
I20260806 09:00:33.180546 10697 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Bootstrap starting.
I20260806 09:00:33.181015 10693 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: No bootstrap required, opened a new log
I20260806 09:00:33.181357 10693 ts_tablet_manager.cc:1397] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260806 09:00:33.184098 10694 tablet_bootstrap.cc:654] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:33.184116 10697 tablet_bootstrap.cc:654] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:33.185506 10697 log.cc:824] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:33.185506 10694 log.cc:824] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:33.190228 10697 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: No bootstrap required, opened a new log
I20260806 09:00:33.190565 10697 ts_tablet_manager.cc:1397] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 09:00:33.190300 10693 raft_consensus.cc:348] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.190745 10693 raft_consensus.cc:374] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:33.190856 10693 raft_consensus.cc:729] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c0f1fbdd444f65b9160913c130bbfc, State: Initialized, Role: FOLLOWER
I20260806 09:00:33.191016 10694 tablet_bootstrap.cc:492] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: No bootstrap required, opened a new log
I20260806 09:00:33.191498 10694 ts_tablet_manager.cc:1397] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 09:00:33.191475 10693 consensus_queue.cc:260] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.192708 10476 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
I20260806 09:00:33.193470 10693 ts_tablet_manager.cc:1428] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260806 09:00:33.202164 10694 raft_consensus.cc:348] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.202109 10697 raft_consensus.cc:348] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.202689 10694 raft_consensus.cc:374] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:33.202682 10697 raft_consensus.cc:374] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:33.202800 10694 raft_consensus.cc:729] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f82ee45bc2f4cd197b5baee374e824f, State: Initialized, Role: FOLLOWER
I20260806 09:00:33.202806 10697 raft_consensus.cc:729] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8285d05e89140fb9d9b1cbb2c4dc004, State: Initialized, Role: FOLLOWER
I20260806 09:00:33.203573 10697 consensus_queue.cc:260] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.203557 10694 consensus_queue.cc:260] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.205296 10664 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
I20260806 09:00:33.205760 10694 ts_tablet_manager.cc:1428] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260806 09:00:33.206869 10697 ts_tablet_manager.cc:1428] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260806 09:00:33.236096 10703 raft_consensus.cc:482] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:33.236469 10703 raft_consensus.cc:504] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.238559 10703 leader_election.cc:288] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047)
I20260806 09:00:33.245584 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.239182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59145 (local address 127.22.98.1:35047)
0806 09:00:33.239409 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.239416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.239441 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:00:33.239640 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.239647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.239728 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.239809 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:33.240241 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.241173 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.243559 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.244662 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.244831 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.244947 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.245193 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.245342 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:00:33.245599 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.238865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.22.98.3:59145)
0806 09:00:33.239159 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.239182 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.239413 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.239864 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.239869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.239900 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:33.240101 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.240112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.241360 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.241364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.243307 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.243325 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.244894 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.244898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.245036 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.245064 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.245144 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.245275 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1}
I20260806 09:00:33.246291 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.239149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:50593 (local address 127.22.98.2:46475)
0806 09:00:33.239409 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.239416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.239976 (+ 560us) server_negotiation.cc:407] Connection header received
0806 09:00:33.240187 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.240191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.240269 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.240365 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:33.240909 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.241746 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.244044 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.245216 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.245347 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.245439 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.245851 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.246011 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 09:00:33.247004 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.238867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46475 (local address 127.22.98.3:50593)
0806 09:00:33.239859 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:33.239876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:33.240060 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:33.240392 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:33.240397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:33.240424 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:33.240753 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.240763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.241922 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.241928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.243884 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:33.243897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.245479 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.245484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:33.245655 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.245687 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:00:33.246480 (+ 793us) client_negotiation.cc:240] Negotiation successful
0806 09:00:33.246628 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":155,"spinlock_wait_cycles":27904,"thread_start_us":383,"threads_started":1}
I20260806 09:00:33.247107 10234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50da3063069f49ddbeff3677cedd1ab6" candidate_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f82ee45bc2f4cd197b5baee374e824f" is_pre_election: true
I20260806 09:00:33.247750 10234 raft_consensus.cc:2454] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8285d05e89140fb9d9b1cbb2c4dc004 in term 0.
I20260806 09:00:33.248060 10417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50da3063069f49ddbeff3677cedd1ab6" candidate_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c0f1fbdd444f65b9160913c130bbfc" is_pre_election: true
I20260806 09:00:33.248435 10509 leader_election.cc:302] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f, b8285d05e89140fb9d9b1cbb2c4dc004; no voters:
I20260806 09:00:33.248510 10417 raft_consensus.cc:2454] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8285d05e89140fb9d9b1cbb2c4dc004 in term 0.
I20260806 09:00:33.248839 10703 raft_consensus.cc:2781] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:00:33.249002 10703 raft_consensus.cc:482] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:00:33.249123 10703 raft_consensus.cc:3037] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:33.251706 10703 raft_consensus.cc:504] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.252339 10703 leader_election.cc:288] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [CANDIDATE]: Term 1 election: Requested vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047)
I20260806 09:00:33.252754 10417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50da3063069f49ddbeff3677cedd1ab6" candidate_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c0f1fbdd444f65b9160913c130bbfc"
I20260806 09:00:33.252925 10417 raft_consensus.cc:3037] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:33.253010 10234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50da3063069f49ddbeff3677cedd1ab6" candidate_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f82ee45bc2f4cd197b5baee374e824f"
I20260806 09:00:33.253237 10234 raft_consensus.cc:3037] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:33.255214 10417 raft_consensus.cc:2454] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8285d05e89140fb9d9b1cbb2c4dc004 in term 1.
I20260806 09:00:33.255496 10234 raft_consensus.cc:2454] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8285d05e89140fb9d9b1cbb2c4dc004 in term 1.
I20260806 09:00:33.255714 10506 leader_election.cc:302] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004; no voters:
I20260806 09:00:33.256108 10703 raft_consensus.cc:2781] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:33.256583 10703 raft_consensus.cc:686] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 LEADER]: Becoming Leader. State: Replica: b8285d05e89140fb9d9b1cbb2c4dc004, State: Running, Role: LEADER
I20260806 09:00:33.257129 10703 consensus_queue.cc:237] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } }
I20260806 09:00:33.262112 10032 catalog_manager.cc:5495] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } health_report { overall_health: UNKNOWN } } }
W20260806 09:00:33.265007 10477 tablet.cc:2369] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:33.284567 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.278727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.22.98.3:45893)
0806 09:00:33.278973 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.278978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.278998 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:00:33.279105 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.279109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.279196 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.279347 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:00:33.279811 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.280573 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.282178 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.282585 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.282691 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.283272 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:00:33.283956 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:00:33.284088 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.284231 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.284309 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 09:00:33.293514 10417 raft_consensus.cc:1261] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Refusing update from remote peer b8285d05e89140fb9d9b1cbb2c4dc004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:33.293514 10234 raft_consensus.cc:1261] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Refusing update from remote peer b8285d05e89140fb9d9b1cbb2c4dc004: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:33.294569 10706 consensus_queue.cc:1035] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:33.294938 10703 consensus_queue.cc:1035] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 09:00:33.304299 10665 tablet.cc:2369] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:33.308753 10709 mvcc.cc:205] Tried to move back new op lower bound from 7315483989159194624 to 7315483989025542144. Current Snapshot: MvccSnapshot[applied={T|T < 7315483989159194624}]
I20260806 09:00:33.310773 10710 mvcc.cc:205] Tried to move back new op lower bound from 7315483989159194624 to 7315483989025542144. Current Snapshot: MvccSnapshot[applied={T|T < 7315483989159194624}]
I20260806 09:00:33.335800 10031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:33.336169 10031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:33.340583 10031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Sending DeleteTablet for 3 replicas of tablet 50da3063069f49ddbeff3677cedd1ab6
I20260806 09:00:33.341883 10392 tablet_service.cc:1514] Processing DeleteTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:33 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:36498
I20260806 09:00:33.341890 10576 tablet_service.cc:1514] Processing DeleteTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:33 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:51972
I20260806 09:00:33.342689 10209 tablet_service.cc:1514] Processing DeleteTablet for tablet 50da3063069f49ddbeff3677cedd1ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:33 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:45918
I20260806 09:00:33.343073 10724 tablet_replica.cc:330] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: stopping tablet replica
I20260806 09:00:33.343318 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 10115
I20260806 09:00:33.343808 10724 raft_consensus.cc:2229] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:00:33.344732 10724 raft_consensus.cc:2258] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:33.346678 10725 tablet_replica.cc:330] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: stopping tablet replica
I20260806 09:00:33.347219 10725 raft_consensus.cc:2229] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:33.347851 10725 raft_consensus.cc:2258] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:33.348763 10727 tablet_replica.cc:330] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: stopping tablet replica
I20260806 09:00:33.348953 10724 ts_tablet_manager.cc:1905] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:33.349201 10727 raft_consensus.cc:2229] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:33.349751 10727 raft_consensus.cc:2258] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:33.352123 10727 ts_tablet_manager.cc:1905] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:33.352210 10725 ts_tablet_manager.cc:1905] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:33.359472 10727 ts_tablet_manager.cc:1918] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:33.359512 10725 ts_tablet_manager.cc:1918] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:33.359784 10725 log.cc:1196] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/50da3063069f49ddbeff3677cedd1ab6
I20260806 09:00:33.359951 10727 log.cc:1196] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/50da3063069f49ddbeff3677cedd1ab6
I20260806 09:00:33.360402 10725 ts_tablet_manager.cc:1939] T 50da3063069f49ddbeff3677cedd1ab6 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting consensus metadata
I20260806 09:00:33.360700 10727 ts_tablet_manager.cc:1939] T 50da3063069f49ddbeff3677cedd1ab6 P 3f82ee45bc2f4cd197b5baee374e824f: Deleting consensus metadata
I20260806 09:00:33.362665 10014 catalog_manager.cc:4841] TS 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047): tablet 50da3063069f49ddbeff3677cedd1ab6 (table test-table [id=8776aaa1df9d44728f26664e2292a86e]) successfully deleted
I20260806 09:00:33.362694 10011 catalog_manager.cc:4841] TS 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): tablet 50da3063069f49ddbeff3677cedd1ab6 (table test-table [id=8776aaa1df9d44728f26664e2292a86e]) successfully deleted
I20260806 09:00:33.363387 10724 ts_tablet_manager.cc:1918] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:33.363613 10724 log.cc:1196] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/50da3063069f49ddbeff3677cedd1ab6
I20260806 09:00:33.364095 10724 ts_tablet_manager.cc:1939] T 50da3063069f49ddbeff3677cedd1ab6 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting consensus metadata
I20260806 09:00:33.365643 10013 catalog_manager.cc:4841] TS b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): tablet 50da3063069f49ddbeff3677cedd1ab6 (table test-table [id=8776aaa1df9d44728f26664e2292a86e]) successfully deleted
Aug 06 09:00:33 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:00:33.374092 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.344758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.22.98.1:35047)
0806 09:00:33.344944 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.344951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.346060 (+ 1109us) server_negotiation.cc:407] Connection header received
0806 09:00:33.346178 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.346184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.346294 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.346400 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:33.352081 (+ 5681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.353110 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.358904 (+ 5794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.359630 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.359767 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.367917 (+ 8150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:33.367933 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:33.367950 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:33.367994 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:33.370985 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.371982 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.371990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.371993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.372135 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.372560 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.372563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.372565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.372987 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:33.373223 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.373586 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.373788 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:00:33.374804 10287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:33.500350 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 10115
I20260806 09:00:33.529768 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 10303
Aug 06 09:00:33 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:00:33.554461 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.530709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.22.98.2:46475)
0806 09:00:33.530920 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.530927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.531064 (+ 137us) server_negotiation.cc:407] Connection header received
0806 09:00:33.531268 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.531273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.531382 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.531493 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:33.533281 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.534507 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.536810 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.537253 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.537357 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.548378 (+ 11021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:33.548408 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:33.548427 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:33.548471 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:33.551485 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.552295 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.552303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.552305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.552442 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.552853 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.552857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.552859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.553261 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:33.553555 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.553905 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.554120 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:00:33.555316 10470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:33.645987 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 10303
I20260806 09:00:33.671644 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 10486
Aug 06 09:00:33 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:00:33.688208 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:33.672502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.22.98.3:45893)
0806 09:00:33.672761 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:33.672767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:33.672787 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:00:33.672860 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:33.672866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:33.672936 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:33.673024 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:33.674278 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.675147 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.676463 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:33.677001 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:33.677121 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:33.681737 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:33.681753 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:33.681772 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:33.681816 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:33.685126 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.685870 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.685878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.685881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.686062 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:33.686490 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:33.686495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:33.686497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:33.686902 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:33.687246 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:33.687526 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 09:00:33.687805 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:00:33.689015 10657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:33.815153 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 10486
I20260806 09:00:33.839107 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 9986
I20260806 09:00:33.840435 10072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:33.951373 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 9986
I20260806 09:00:33.977965 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--webserver_interface=127.22.98.62
--webserver_port=41591
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:34.335219 10735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:34.335588 10735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:34.335755 10735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:34.343277 10735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:34.343436 10735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:34.343520 10735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:34.343621 10735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:34.343694 10735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:34.354349 10735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=41591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:34.355830 10735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:34.357276 10735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:34.382763 10750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:34.382826 10746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:34.382793 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:34.383697 10735 server_base.cc:1034] running on GCE node
I20260806 09:00:34.384632 10735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:34.386324 10735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:34.387595 10735 hybrid_clock.cc:648] HybridClock initialized: now 1786006834387502 us; error 46 us; skew 500 ppm
I20260806 09:00:34.388051 10735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:34 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:34.394490 10735 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:34.396359 10735 webserver.cc:460] Webserver started at http://127.22.98.62:41591/ using document root <none> and password file <none>
I20260806 09:00:34.397169 10735 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:34.397315 10735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:34.404120 10735 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:00:34.407486 10759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:34.409145 10735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:00:34.409327 10735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "0e8031c1a8d8486ca3427a55a011c8da"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "4e0c48bf16ab4ebfc4a397c890667412"
server_key_iv: "f7a4b4f068f5eb1c38cf16e38b330b9e"
server_key_version: "encryptionkey@0"
I20260806 09:00:34.410466 10735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:34.431926 10735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:34.436908 10735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:34.437227 10735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:34.456856 10735 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:43991
I20260806 09:00:34.456882 10829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:43991 every 8 connection(s)
I20260806 09:00:34.458478 10735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:34.465888 10830 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:00:34.467156 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 10735
I20260806 09:00:34.467981 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:35047
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=42289
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:34.469743 10830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap starting.
I20260806 09:00:34.491345 10830 log.cc:824] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:34.504226 10830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:00:34.504771 10830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap complete.
I20260806 09:00:34.514001 10830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:34.515034 10830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Initialized, Role: FOLLOWER
I20260806 09:00:34.515743 10830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:34.515995 10830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:34.516122 10830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:34.516283 10830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:00:34.520540 10830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:34.521184 10830 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e8031c1a8d8486ca3427a55a011c8da; no voters:
I20260806 09:00:34.521845 10830 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:00:34.521993 10835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:00:34.523285 10835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 LEADER]: Becoming Leader. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Running, Role: LEADER
I20260806 09:00:34.523993 10835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:34.524348 10830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:34.526985 10837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8031c1a8d8486ca3427a55a011c8da. Latest consensus state: current_term: 2 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:34.527406 10837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:34.528475 10836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:34.528776 10836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:34.536875 10848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:34.546263 10848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:34.548293 10848 catalog_manager.cc:1259] Loaded cluster ID: 8f0c54db65824e94b4e705b2a99d3ef8
I20260806 09:00:34.548508 10848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:34.552189 10848 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 09:00:34.573796 10848 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 09:00:34.583936 10848 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:34.585911 10848 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Loaded TSK: 0
I20260806 09:00:34.587471 10848 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:00:34.845395 10833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:34.845732 10833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:34.845861 10833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:34.853546 10833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:34.853686 10833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:34.853837 10833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:34.865005 10833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:35047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=42289
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:34.866698 10833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:34.868544 10833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:34.885095 10873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:34.885090 10872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:34.885826 10878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:34.886322 10833 server_base.cc:1034] running on GCE node
I20260806 09:00:34.887045 10833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:34.888167 10833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:34.889401 10833 hybrid_clock.cc:648] HybridClock initialized: now 1786006834889330 us; error 60 us; skew 500 ppm
I20260806 09:00:34.889776 10833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:34 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:34.893314 10833 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:34.894956 10833 webserver.cc:460] Webserver started at http://127.22.98.1:42289/ using document root <none> and password file <none>
I20260806 09:00:34.895763 10833 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:34.895910 10833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:34.901723 10833 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 09:00:34.904808 10888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:34.906432 10833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:00:34.906621 10833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "3f82ee45bc2f4cd197b5baee374e824f"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "a95ec2051d9c55d4d77223963b78c437"
server_key_iv: "ce111c996712664ab36ad19fbb2b8352"
server_key_version: "encryptionkey@0"
I20260806 09:00:34.907734 10833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:34.929667 10833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:34.934407 10833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:34.934684 10833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:34.935796 10833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:34.937318 10833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:34.937425 10833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:34.937544 10833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:34.937629 10833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:34.969242 10833 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:35047
I20260806 09:00:34.969313 11053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:35047 every 8 connection(s)
I20260806 09:00:34.970705 10833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:34.978554 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 10833
I20260806 09:00:34.979146 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:46475
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=35843
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:34 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006834, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:34.995939 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:34.973608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:49429 (local address 127.22.98.62:43991)
0806 09:00:34.974200 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:34.974212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:34.974878 (+ 666us) server_negotiation.cc:407] Connection header received
0806 09:00:34.977030 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:34.977042 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:34.977384 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:34.977717 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:34.979395 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:34.981302 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:34.982869 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:34.983843 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:34.984021 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:34.987367 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:34.987376 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:34.987392 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:34.987434 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:34.991217 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:34.992031 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:34.992038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:34.992044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:34.992215 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:34.992646 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:34.992650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:34.992652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:34.993157 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:34.993366 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:34.994151 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 09:00:34.994573 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":256,"threads_started":1}
I20260806 09:00:34.995939 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:34.973713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:49429)
0806 09:00:34.974691 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:34.974735 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:34.976478 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:34.977896 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:34.977908 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:34.978548 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:34.979139 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:34.979158 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:34.981463 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:34.981467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:34.982707 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:34.982719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:34.984066 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:34.984072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:34.984263 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:34.984895 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:34.984923 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:34.987087 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:34.991366 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:34.991376 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:34.991394 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:34.991894 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:34.992338 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:34.992343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:34.992348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:34.992538 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:34.993305 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:34.993314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:34.993653 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 09:00:34.993914 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 09:00:34.994510 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":545,"threads_started":1}
I20260806 09:00:34.999034 11054 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:34.999452 11054 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:35.000550 11054 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:35.003211 10783 ts_manager.cc:194] Registered new tserver with Master: 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047)
I20260806 09:00:35.004808 10783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:49429
W20260806 09:00:35.335875 11062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:35.336198 11062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:35.336342 11062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:35.343894 11062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:35.344025 11062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:35.344226 11062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:35.354569 11062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:46475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=35843
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:35.356016 11062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:35.357419 11062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:35.371974 11070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:35.371995 11072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:35.372537 11067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:35.372625 11062 server_base.cc:1034] running on GCE node
I20260806 09:00:35.373340 11062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:35.374420 11062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:35.375631 11062 hybrid_clock.cc:648] HybridClock initialized: now 1786006835375570 us; error 50 us; skew 500 ppm
I20260806 09:00:35.375995 11062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:35 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006835, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:35.379182 11062 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:35.380699 11062 webserver.cc:460] Webserver started at http://127.22.98.2:35843/ using document root <none> and password file <none>
I20260806 09:00:35.381387 11062 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:35.381523 11062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:35.387056 11062 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 09:00:35.389927 11081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.391366 11062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:00:35.391521 11062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "44c0f1fbdd444f65b9160913c130bbfc"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "3fb4d5379a70cb48846e81f08106a4b0"
server_key_iv: "94ea620b1768e7266f6199ca0cd2842e"
server_key_version: "encryptionkey@0"
I20260806 09:00:35.392530 11062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:35.429809 11062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:35.434846 11062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:35.435115 11062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:35.436235 11062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:35.437633 11062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:35.437726 11062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.437834 11062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:35.437901 11062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.470098 11062 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46475
I20260806 09:00:35.470186 11238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46475 every 8 connection(s)
I20260806 09:00:35.471669 11062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:35.472885 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 11062
I20260806 09:00:35.473675 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:45893
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=43137
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:00:35 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006835, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:35.495586 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.475972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:37623 (local address 127.22.98.62:43991)
0806 09:00:35.476195 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:35.476202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:35.476739 (+ 537us) server_negotiation.cc:407] Connection header received
0806 09:00:35.479147 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:35.479153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:35.479281 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:35.479395 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:35.481599 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.482584 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.484524 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.485113 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.485239 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:35.489389 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:35.489395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:35.489398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:35.489435 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:35.492188 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.493139 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.493143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.493145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.493279 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.493730 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.493733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.493735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.493992 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:35.494110 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:35.495065 (+ 955us) server_negotiation.cc:299] Negotiation successful
0806 09:00:35.495263 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:00:35.497118 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.475270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:37623)
0806 09:00:35.476544 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:35.476595 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:35.478801 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:35.479945 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:35.479959 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:35.480675 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:35.481356 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:35.481377 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.482806 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.482811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:35.484330 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:35.484348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.485352 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.485358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:35.485634 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:35.486388 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:35.486415 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:35.488983 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:35.492337 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:35.492347 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:35.492367 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:35.493023 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:35.493375 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:35.493379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:35.493383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:35.493637 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:35.494141 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:35.494150 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:35.494596 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 09:00:35.494901 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 09:00:35.495471 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"thread_start_us":810,"threads_started":1}
I20260806 09:00:35.498773 11239 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:35.499138 11239 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:35.500173 11239 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:35.501968 10783 ts_manager.cc:194] Registered new tserver with Master: 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475)
I20260806 09:00:35.503144 10783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:37623
W20260806 09:00:35.832098 11242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:35.832396 11242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:35.832489 11242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:35.839845 11242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:35.839989 11242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:35.840117 11242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:00:35.850215 11242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:45893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=43137
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:35.851578 11242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:35.853039 11242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:35.866185 11258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:35.866276 11250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:35.866259 11249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:35.866918 11242 server_base.cc:1034] running on GCE node
I20260806 09:00:35.867570 11242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:35.868553 11242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:35.869767 11242 hybrid_clock.cc:648] HybridClock initialized: now 1786006835869710 us; error 47 us; skew 500 ppm
I20260806 09:00:35.870090 11242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:35 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006835, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:35.873315 11242 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:35.874792 11242 webserver.cc:460] Webserver started at http://127.22.98.3:43137/ using document root <none> and password file <none>
I20260806 09:00:35.875490 11242 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:35.875662 11242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:35.881940 11242 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 09:00:35.885172 11266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.886654 11242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:00:35.886862 11242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "0082fc114cce70f5871e9e44736ab002"
server_key_iv: "14e7425e86bb885bb1c572fa1fa804a4"
server_key_version: "encryptionkey@0"
I20260806 09:00:35.887960 11242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:35.908896 11242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:35.913803 11242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:35.914111 11242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:35.915314 11242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:35.916869 11242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:35.916973 11242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.917066 11242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:35.917121 11242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:35.950749 11242 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:45893
I20260806 09:00:35.950830 11425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:45893 every 8 connection(s)
I20260806 09:00:35.952431 11242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:35.960979 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 11242
Aug 06 09:00:35 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006835, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:35.975570 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.955247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:45957 (local address 127.22.98.62:43991)
0806 09:00:35.955414 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:35.955418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:35.956772 (+ 1354us) server_negotiation.cc:407] Connection header received
0806 09:00:35.958738 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:35.958745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:35.958835 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:35.958930 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:35.960839 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.961624 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.964020 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.964495 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.964592 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:35.969270 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:35.969275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:35.969278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:35.969312 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:35.971905 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.972650 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.972654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.972657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.972808 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.973366 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.973371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.973375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.973772 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:35.973927 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:35.974959 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0806 09:00:35.975281 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:00:35.976599 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.955774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:45957)
0806 09:00:35.956551 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:35.956597 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:35.958412 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:35.959244 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:35.959253 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:35.959925 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:35.960613 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:35.960634 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.961859 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.961866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:35.963855 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:35.963874 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.964753 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.964762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:35.965067 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:35.965749 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:35.965779 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:35.969023 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:35.972044 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:35.972054 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:35.972071 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:35.972507 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:35.972935 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:35.972941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:35.972944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:35.973219 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:35.974011 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:35.974019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:35.974476 (+ 457us) client_negotiation.cc:769] Sending connection context
0806 09:00:35.974803 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 09:00:35.975371 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
I20260806 09:00:35.978449 11426 heartbeater.cc:347] Connected to a master server at 127.22.98.62:43991
I20260806 09:00:35.978638 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.962086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.22.98.62:43991)
0806 09:00:35.962938 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:35.962943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:35.962961 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:00:35.963005 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:35.963008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:35.963075 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:35.963160 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:35.965048 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.965786 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.967607 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.968277 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.968380 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:35.972159 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:35.972166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:35.972171 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:35.972212 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:35.975657 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.976421 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.976426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.976429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.976579 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:35.977089 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:35.977093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:35.977095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:35.977451 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:35.977678 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:35.978084 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 09:00:35.978273 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":637,"threads_started":1}
I20260806 09:00:35.978816 11426 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:35.979760 11426 heartbeater.cc:511] Master 127.22.98.62:43991 requested a full tablet report, sending...
I20260806 09:00:35.981415 10783 ts_manager.cc:194] Registered new tserver with Master: b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
I20260806 09:00:35.982214 10783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:45957
I20260806 09:00:35.994379 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:00:36.002388 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:35.995448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.22.98.62:43991)
0806 09:00:35.995671 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:35.995676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:35.996070 (+ 394us) server_negotiation.cc:407] Connection header received
0806 09:00:35.996386 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:35.996391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:35.996568 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:35.996726 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:00:35.997174 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:35.998111 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:35.999772 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.000288 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.000417 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.001066 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:00:36.001694 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:00:36.001866 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.001975 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.002081 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:00:36.008970 11054 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
I20260806 09:00:36.003707 10783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:00:36.012001 10783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:00:36.047051 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.025559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45930)
0806 09:00:36.038336 (+ 12777us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.038359 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:36.038857 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:36.039992 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:36.039996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:36.040021 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:36.040435 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.040452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.042080 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.042086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.044249 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.044263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.045930 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.045937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.046158 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.046213 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 09:00:36.046358 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 09:00:36.046580 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12605,"mutex_wait_us":46,"thread_start_us":11923,"threads_started":1}
I20260806 09:00:36.047674 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.026177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.22.98.2:46475)
0806 09:00:36.026863 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.026874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.038419 (+ 11545us) server_negotiation.cc:407] Connection header received
0806 09:00:36.038560 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.038569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.038917 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.040154 (+ 1237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:36.040432 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.041980 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.044623 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.046128 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.046281 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.046480 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.046771 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.047023 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":323,"threads_started":1}
I20260806 09:00:36.047729 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.026045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.22.98.1:35047)
0806 09:00:36.036956 (+ 10911us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.036969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.038505 (+ 1536us) server_negotiation.cc:407] Connection header received
0806 09:00:36.039112 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.039123 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.039394 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.039616 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:36.040667 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.041924 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.044429 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.045696 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.045831 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.046105 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.047274 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.047427 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10627,"thread_start_us":10949,"threads_started":1}
I20260806 09:00:36.049012 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.028188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45893 (local address 127.0.0.1:51998)
0806 09:00:36.039696 (+ 11508us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.039726 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:36.039868 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:36.040983 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:36.040988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:36.041041 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:36.041481 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.041495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.043647 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.043653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.046264 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.046289 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.048210 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.048218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.048387 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.048425 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:00:36.048508 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 09:00:36.048638 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11325,"thread_start_us":11123,"threads_started":1}
I20260806 09:00:36.049507 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.029181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.22.98.3:45893)
0806 09:00:36.039081 (+ 9900us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.039091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.040025 (+ 934us) server_negotiation.cc:407] Connection header received
0806 09:00:36.040162 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.040170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.040437 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.040651 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:36.041723 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.043438 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.046485 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.047998 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.048123 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.048375 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.048988 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.049174 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9698,"thread_start_us":9533,"threads_started":1}
I20260806 09:00:36.047117 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.025495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46475 (local address 127.0.0.1:36526)
0806 09:00:36.037977 (+ 12482us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.038047 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:36.038321 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:36.039233 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:36.039246 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:36.039285 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:36.039885 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.039904 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.042178 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.042184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.044422 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.044436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.046373 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.046380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.046561 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.046592 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:00:36.046683 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 09:00:36.046835 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12172,"thread_start_us":13034,"threads_started":1}
I20260806 09:00:36.052456 10958 tablet_service.cc:1467] Processing CreateTablet for tablet fbc7cd301895472f9783ba7e4d57f179 (DEFAULT_TABLE table=test-table [id=d73876489a9643ec89de4621a169bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:36.053462 11146 tablet_service.cc:1467] Processing CreateTablet for tablet fbc7cd301895472f9783ba7e4d57f179 (DEFAULT_TABLE table=test-table [id=d73876489a9643ec89de4621a169bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:36.053462 11337 tablet_service.cc:1467] Processing CreateTablet for tablet fbc7cd301895472f9783ba7e4d57f179 (DEFAULT_TABLE table=test-table [id=d73876489a9643ec89de4621a169bc77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:36.054073 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbc7cd301895472f9783ba7e4d57f179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:36.054915 11337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbc7cd301895472f9783ba7e4d57f179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:36.054924 11146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbc7cd301895472f9783ba7e4d57f179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:36.069970 11453 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Bootstrap starting.
I20260806 09:00:36.072314 11455 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Bootstrap starting.
I20260806 09:00:36.074285 11453 tablet_bootstrap.cc:654] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:36.075762 11453 log.cc:824] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:36.076203 11455 tablet_bootstrap.cc:654] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:36.076517 11454 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Bootstrap starting.
I20260806 09:00:36.077633 11455 log.cc:824] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:36.080651 11454 tablet_bootstrap.cc:654] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:36.081033 11453 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: No bootstrap required, opened a new log
I20260806 09:00:36.081251 11455 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: No bootstrap required, opened a new log
I20260806 09:00:36.081353 11453 ts_tablet_manager.cc:1397] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260806 09:00:36.081547 11455 ts_tablet_manager.cc:1397] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 09:00:36.082118 11454 log.cc:824] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:36.085837 11454 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: No bootstrap required, opened a new log
I20260806 09:00:36.086174 11454 ts_tablet_manager.cc:1397] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 09:00:36.090624 11453 raft_consensus.cc:348] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.091149 11453 raft_consensus.cc:374] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:36.091259 11453 raft_consensus.cc:729] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f82ee45bc2f4cd197b5baee374e824f, State: Initialized, Role: FOLLOWER
I20260806 09:00:36.091960 11453 consensus_queue.cc:260] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.092067 11455 raft_consensus.cc:348] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.092653 11455 raft_consensus.cc:374] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:36.092758 11455 raft_consensus.cc:729] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c0f1fbdd444f65b9160913c130bbfc, State: Initialized, Role: FOLLOWER
I20260806 09:00:36.093556 11455 consensus_queue.cc:260] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.094105 11453 ts_tablet_manager.cc:1428] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260806 09:00:36.096330 11239 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
I20260806 09:00:36.097133 11455 ts_tablet_manager.cc:1428] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260806 09:00:36.097756 11454 raft_consensus.cc:348] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.098243 11454 raft_consensus.cc:374] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:36.098343 11454 raft_consensus.cc:729] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8285d05e89140fb9d9b1cbb2c4dc004, State: Initialized, Role: FOLLOWER
I20260806 09:00:36.099009 11454 consensus_queue.cc:260] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.100255 11426 heartbeater.cc:503] Master 127.22.98.62:43991 was elected leader, sending a full tablet report...
I20260806 09:00:36.100807 11454 ts_tablet_manager.cc:1428] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
W20260806 09:00:36.204293 11427 tablet.cc:2369] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:00:36.223158 11055 tablet.cc:2369] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:00:36.230777 11240 tablet.cc:2369] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:36.244151 11459 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:36.244519 11459 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.246448 11459 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
I20260806 09:00:36.255120 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.247012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:38993 (local address 127.22.98.2:46475)
0806 09:00:36.247262 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.247270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.247831 (+ 561us) server_negotiation.cc:407] Connection header received
0806 09:00:36.248084 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.248090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.248163 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.248292 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:36.249234 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.250412 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.252595 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.253904 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.254067 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.254186 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.254705 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.254883 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:00:36.255126 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.247030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:51357 (local address 127.22.98.3:45893)
0806 09:00:36.247242 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.247249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.247965 (+ 716us) server_negotiation.cc:407] Connection header received
0806 09:00:36.248070 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.248075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.248164 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.248300 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:36.249265 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.250432 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.252631 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.254050 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.254171 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.254285 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.254691 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.254842 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:00:36.255345 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.246884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46475 (local address 127.22.98.1:38993)
0806 09:00:36.247679 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.247702 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:36.247889 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:36.248381 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:36.248387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:36.248452 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:36.249013 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.249028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.250560 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.250564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.252392 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.252409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.254191 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.254199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.254455 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.254517 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 09:00:36.254633 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:00:36.254826 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":166,"spinlock_wait_cycles":6912,"thread_start_us":393,"threads_started":1}
I20260806 09:00:36.255365 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.246938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45893 (local address 127.22.98.1:51357)
0806 09:00:36.247830 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.247847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:36.247959 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:36.248381 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:36.248387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:36.248458 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:36.249017 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.249028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.250622 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.250627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.252390 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:36.252409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.254264 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.254269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:36.254462 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.254517 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 09:00:36.254633 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:00:36.254826 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":68,"spinlock_wait_cycles":55296,"thread_start_us":513,"threads_started":1}
I20260806 09:00:36.256546 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbc7cd301895472f9783ba7e4d57f179" candidate_uuid: "3f82ee45bc2f4cd197b5baee374e824f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" is_pre_election: true
I20260806 09:00:36.256603 11177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbc7cd301895472f9783ba7e4d57f179" candidate_uuid: "3f82ee45bc2f4cd197b5baee374e824f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c0f1fbdd444f65b9160913c130bbfc" is_pre_election: true
I20260806 09:00:36.257118 11365 raft_consensus.cc:2454] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f82ee45bc2f4cd197b5baee374e824f in term 0.
I20260806 09:00:36.257108 11177 raft_consensus.cc:2454] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f82ee45bc2f4cd197b5baee374e824f in term 0.
I20260806 09:00:36.257843 10889 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f, 44c0f1fbdd444f65b9160913c130bbfc; no voters:
I20260806 09:00:36.258255 11459 raft_consensus.cc:2781] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:00:36.258423 11459 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:00:36.258543 11459 raft_consensus.cc:3037] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:36.260897 11459 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.261605 11459 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 1 election: Requested vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
I20260806 09:00:36.262199 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbc7cd301895472f9783ba7e4d57f179" candidate_uuid: "3f82ee45bc2f4cd197b5baee374e824f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
I20260806 09:00:36.262205 11177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbc7cd301895472f9783ba7e4d57f179" candidate_uuid: "3f82ee45bc2f4cd197b5baee374e824f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c0f1fbdd444f65b9160913c130bbfc"
I20260806 09:00:36.262413 11177 raft_consensus.cc:3037] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:36.262425 11365 raft_consensus.cc:3037] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:36.264988 11365 raft_consensus.cc:2454] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f82ee45bc2f4cd197b5baee374e824f in term 1.
I20260806 09:00:36.264986 11177 raft_consensus.cc:2454] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f82ee45bc2f4cd197b5baee374e824f in term 1.
I20260806 09:00:36.265465 10890 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f, b8285d05e89140fb9d9b1cbb2c4dc004; no voters:
I20260806 09:00:36.265810 11459 raft_consensus.cc:2781] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:36.266270 11459 raft_consensus.cc:686] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 LEADER]: Becoming Leader. State: Replica: 3f82ee45bc2f4cd197b5baee374e824f, State: Running, Role: LEADER
I20260806 09:00:36.266753 11459 consensus_queue.cc:237] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:36.271451 10783 catalog_manager.cc:5495] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1). New cstate: current_term: 1 leader_uuid: "3f82ee45bc2f4cd197b5baee374e824f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } health_report { overall_health: UNKNOWN } } }
I20260806 09:00:36.291190 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.285630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.22.98.1:35047)
0806 09:00:36.285857 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.285863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.285883 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:00:36.286076 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.286080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.286188 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.286332 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:00:36.286737 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.287461 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.289024 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.289437 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.289548 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.290083 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:00:36.290598 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:00:36.290750 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.290842 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.290925 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:00:36.300133 11365 raft_consensus.cc:1261] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Refusing update from remote peer 3f82ee45bc2f4cd197b5baee374e824f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:36.300145 11177 raft_consensus.cc:1261] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Refusing update from remote peer 3f82ee45bc2f4cd197b5baee374e824f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:36.301106 11468 consensus_queue.cc:1035] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:36.301458 11459 consensus_queue.cc:1035] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:36.328380 10782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:36.328670 10782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:36.332378 10782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Sending DeleteTablet for 3 replicas of tablet fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:36.333721 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet fbc7cd301895472f9783ba7e4d57f179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:36 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:45930
I20260806 09:00:36.334081 11337 tablet_service.cc:1514] Processing DeleteTablet for tablet fbc7cd301895472f9783ba7e4d57f179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:36 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:51998
I20260806 09:00:36.334270 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 10833
I20260806 09:00:36.334517 11146 tablet_service.cc:1514] Processing DeleteTablet for tablet fbc7cd301895472f9783ba7e4d57f179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:36 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:36526
I20260806 09:00:36.335425 11481 tablet_replica.cc:330] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: stopping tablet replica
I20260806 09:00:36.336030 11483 tablet_replica.cc:330] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: stopping tablet replica
I20260806 09:00:36.336618 11481 raft_consensus.cc:2229] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:00:36.337219 11483 raft_consensus.cc:2229] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:36.337967 11483 raft_consensus.cc:2258] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:36.338958 11481 raft_consensus.cc:2258] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:36.341578 11483 ts_tablet_manager.cc:1905] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:36.349212 11481 ts_tablet_manager.cc:1905] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:36.352165 11482 tablet_replica.cc:330] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: stopping tablet replica
I20260806 09:00:36.352659 11482 raft_consensus.cc:2229] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:36.352914 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.335128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.22.98.1:35047)
0806 09:00:36.335331 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.335337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.335465 (+ 128us) server_negotiation.cc:407] Connection header received
0806 09:00:36.335716 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.335721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.335804 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.335928 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:36.337768 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.338988 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.340895 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.341552 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.341693 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.345241 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:36.345262 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:36.345281 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:36.345322 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:36.349475 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.350435 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.350445 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.350449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.350655 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.351095 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.351099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.351102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.351618 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:36.351975 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.352290 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.352547 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:00:36.353296 11482 raft_consensus.cc:2258] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:36.353672 11048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:36.355413 11483 ts_tablet_manager.cc:1918] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:36.355613 11483 log.cc:1196] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:36.355893 11482 ts_tablet_manager.cc:1905] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:36.356236 11483 ts_tablet_manager.cc:1939] T fbc7cd301895472f9783ba7e4d57f179 P 44c0f1fbdd444f65b9160913c130bbfc: Deleting consensus metadata
I20260806 09:00:36.358379 10760 catalog_manager.cc:4841] TS 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): tablet fbc7cd301895472f9783ba7e4d57f179 (table test-table [id=d73876489a9643ec89de4621a169bc77]) successfully deleted
I20260806 09:00:36.362811 11482 ts_tablet_manager.cc:1918] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:36.363013 11482 log.cc:1196] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:36.363487 11482 ts_tablet_manager.cc:1939] T fbc7cd301895472f9783ba7e4d57f179 P b8285d05e89140fb9d9b1cbb2c4dc004: Deleting consensus metadata
I20260806 09:00:36.365206 10761 catalog_manager.cc:4841] TS b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): tablet fbc7cd301895472f9783ba7e4d57f179 (table test-table [id=d73876489a9643ec89de4621a169bc77]) successfully deleted
I20260806 09:00:36.455371 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 10833
I20260806 09:00:36.486097 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 11062
W20260806 09:00:36.486006 10762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:35047 (error 108)
I20260806 09:00:36.488310 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.487891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45962)
0806 09:00:36.488052 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.488158 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 09:00:36.488565 10762 catalog_manager.cc:4584] TS 3f82ee45bc2f4cd197b5baee374e824f (127.22.98.1:35047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbc7cd301895472f9783ba7e4d57f179: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
I20260806 09:00:36.500197 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.486945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.22.98.2:46475)
0806 09:00:36.487190 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.487197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.487290 (+ 93us) server_negotiation.cc:407] Connection header received
0806 09:00:36.487491 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.487497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.487621 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.487741 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:36.489459 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.490316 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.491837 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.492230 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.492327 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.494927 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:36.494941 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:36.494957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:36.494989 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:36.497689 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.498384 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.498392 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.498394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.498506 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.498830 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.498833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.498835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.499208 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:36.499418 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.499675 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.499861 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 09:00:36.500850 11233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:36.542639 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.542086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45968)
0806 09:00:36.542351 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.542465 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 09:00:36.543723 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.543331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45980)
0806 09:00:36.543481 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.543594 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 09:00:36.590173 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 11062
I20260806 09:00:36.604282 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.603760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:45990)
0806 09:00:36.603998 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.604096 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 09:00:36.605366 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.605000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:46004)
0806 09:00:36.605169 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.605243 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 09:00:36.614123 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 11242
I20260806 09:00:36.627291 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.614878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.22.98.3:45893)
0806 09:00:36.615042 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:36.615048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:36.615070 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:00:36.615282 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:36.615287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:36.615362 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:36.615452 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:36.616820 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.617523 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.618866 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:36.619259 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:36.619351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:36.621975 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:36.621991 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:36.622009 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:36.622051 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:36.624658 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.625357 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.625365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.625367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.625519 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:36.625937 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:36.625940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:36.625942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:36.626349 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:36.626608 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:36.626796 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 09:00:36.626961 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 09:00:36.627992 11415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:36.686209 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.685668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:46012)
0806 09:00:36.685913 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.686018 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 09:00:36.687372 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:36.686960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:35047 (local address 127.0.0.1:46024)
0806 09:00:36.687170 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:36.687243 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:35047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 09:00:36.717396 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 11242
I20260806 09:00:36.739620 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 10735
I20260806 09:00:36.740885 10820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:36.855872 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 10735
I20260806 09:00:36.883802 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--webserver_interface=127.22.98.62
--webserver_port=41591
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:37.236698 11491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:37.237036 11491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:37.237160 11491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:37.245965 11491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:37.246102 11491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:37.246163 11491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:37.246248 11491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:37.246320 11491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:37.256851 11491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=41591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:37.258354 11491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:37.259824 11491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:37.271993 11498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:37.271970 11501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:37.272030 11499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:37.272943 11491 server_base.cc:1034] running on GCE node
I20260806 09:00:37.273820 11491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:37.275350 11491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:37.276566 11491 hybrid_clock.cc:648] HybridClock initialized: now 1786006837276500 us; error 54 us; skew 500 ppm
I20260806 09:00:37.276923 11491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:37 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006837, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:37.280313 11491 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:37.282172 11491 webserver.cc:460] Webserver started at http://127.22.98.62:41591/ using document root <none> and password file <none>
I20260806 09:00:37.282950 11491 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:37.283087 11491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:37.289345 11491 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 09:00:37.292579 11510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:37.293967 11491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:00:37.294132 11491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "0e8031c1a8d8486ca3427a55a011c8da"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "4e0c48bf16ab4ebfc4a397c890667412"
server_key_iv: "f7a4b4f068f5eb1c38cf16e38b330b9e"
server_key_version: "encryptionkey@0"
I20260806 09:00:37.295148 11491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:37.332652 11491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:37.336830 11491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:37.337100 11491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:37.356194 11491 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:43991
I20260806 09:00:37.356232 11587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:43991 every 8 connection(s)
I20260806 09:00:37.357889 11491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:37.363163 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 11491
I20260806 09:00:37.364102 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:35047
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=42289
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:00:37.365694 11588 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:00:37.369335 11588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap starting.
I20260806 09:00:37.390257 11588 log.cc:824] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:37.410071 11588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:00:37.410635 11588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da: Bootstrap complete.
I20260806 09:00:37.419953 11588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:37.420877 11588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Initialized, Role: FOLLOWER
I20260806 09:00:37.421583 11588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:37.421830 11588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:37.421947 11588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:37.422108 11588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:00:37.426323 11588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:37.426893 11588 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e8031c1a8d8486ca3427a55a011c8da; no voters:
I20260806 09:00:37.427627 11588 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 09:00:37.427769 11594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 3 FOLLOWER]: Leader election won for term 3
I20260806 09:00:37.428934 11594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [term 3 LEADER]: Becoming Leader. State: Replica: 0e8031c1a8d8486ca3427a55a011c8da, State: Running, Role: LEADER
I20260806 09:00:37.429582 11594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } }
I20260806 09:00:37.430063 11588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:37.432227 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8031c1a8d8486ca3427a55a011c8da. Latest consensus state: current_term: 3 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:37.432631 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:37.433244 11596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e8031c1a8d8486ca3427a55a011c8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8031c1a8d8486ca3427a55a011c8da" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 43991 } } }
I20260806 09:00:37.433526 11596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8031c1a8d8486ca3427a55a011c8da [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:37.445125 11604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:37.456007 11604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:37.457751 11604 catalog_manager.cc:1259] Loaded cluster ID: 8f0c54db65824e94b4e705b2a99d3ef8
I20260806 09:00:37.457973 11604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 09:00:37.460526 11604 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f729c1379ad google::LogMessage::Fail() at ??:0
@ 0x7f729c13a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f729c1374de google::LogMessage::Flush() at ??:0
@ 0x7f729c138a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f72b1b6a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f72b1b67976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f72b1bef610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f72b1bef439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f729ed83345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f729ed8691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f729ed86799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f729ed61eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f72a203b6db start_thread at ??:0
@ 0x7f729afd161f clone at ??:0
W20260806 09:00:37.790485 11591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:37.790798 11591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:37.790930 11591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:37.798440 11591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:37.798570 11591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:37.798729 11591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:37.809029 11591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:35047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=42289
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:37.810420 11591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:37.811834 11591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:37.826668 11650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:37.826778 11651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:37.827056 11653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:37.827798 11591 server_base.cc:1034] running on GCE node
I20260806 09:00:37.828431 11591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:37.829375 11591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:37.830585 11591 hybrid_clock.cc:648] HybridClock initialized: now 1786006837830514 us; error 64 us; skew 500 ppm
I20260806 09:00:37.830919 11591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:37 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006837, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:37.834097 11591 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:37.835486 11591 webserver.cc:460] Webserver started at http://127.22.98.1:42289/ using document root <none> and password file <none>
I20260806 09:00:37.836220 11591 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:37.836329 11591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:37.842123 11591 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 09:00:37.845564 11661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:37.847067 11591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:00:37.847229 11591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "3f82ee45bc2f4cd197b5baee374e824f"
format_stamp: "Formatted at 2026-08-06 09:00:31 on dist-test-slave-jpj1"
server_key: "a95ec2051d9c55d4d77223963b78c437"
server_key_iv: "ce111c996712664ab36ad19fbb2b8352"
server_key_version: "encryptionkey@0"
I20260806 09:00:37.848268 11591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:37.874794 11591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:37.879484 11591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:37.879824 11591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:37.881006 11591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:37.883080 11671 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 09:00:37.889500 11591 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 09:00:37.889714 11591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 09:00:37.889861 11591 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 09:00:37.893348 11591 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 09:00:37.893458 11591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260806 09:00:37.893873 11671 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Bootstrap starting.
I20260806 09:00:37.903851 11671 log.cc:824] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:37.926047 11671 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:00:37.927053 11671 tablet_bootstrap.cc:492] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Bootstrap complete.
I20260806 09:00:37.927825 11671 ts_tablet_manager.cc:1397] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent bootstrapping tablet: real 0.034s user 0.029s sys 0.004s
I20260806 09:00:37.932713 11591 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:35047
I20260806 09:00:37.932720 11820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:35047 every 8 connection(s)
I20260806 09:00:37.934490 11591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:37.938023 11671 raft_consensus.cc:348] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:37.939136 11671 raft_consensus.cc:729] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f82ee45bc2f4cd197b5baee374e824f, State: Initialized, Role: FOLLOWER
I20260806 09:00:37.939831 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:37.937475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:56545)
0806 09:00:37.938385 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:37.938723 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":370,"threads_started":1}
I20260806 09:00:37.939798 11671 consensus_queue.cc:260] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
W20260806 09:00:37.940517 11821 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:43991 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:43991: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
I20260806 09:00:37.941514 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:37.941165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:45547)
0806 09:00:37.941316 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:37.941392 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 09:00:37.941541 11671 ts_tablet_manager.cc:1428] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 09:00:37.942444 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:37.942091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:42321)
0806 09:00:37.942246 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:37.942321 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 09:00:37.942679 11821 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:00:37.944931 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 11591
I20260806 09:00:37.945514 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:46475
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=35843
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:38.301982 11825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:38.302285 11825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:38.302383 11825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:38.310079 11825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:38.310194 11825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:38.310310 11825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:38.320184 11825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:46475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=35843
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:38.321626 11825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:38.323261 11825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:38.336601 11832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:38.336583 11835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:38.336536 11833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:38.337385 11825 server_base.cc:1034] running on GCE node
I20260806 09:00:38.338034 11825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:38.338961 11825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:38.340174 11825 hybrid_clock.cc:648] HybridClock initialized: now 1786006838340121 us; error 39 us; skew 500 ppm
I20260806 09:00:38.340507 11825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:38 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006838, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:38.343706 11825 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:38.345221 11825 webserver.cc:460] Webserver started at http://127.22.98.2:35843/ using document root <none> and password file <none>
I20260806 09:00:38.345935 11825 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:38.346060 11825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:38.352228 11825 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:00:38.355245 11844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:38.356695 11825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:38.356906 11825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "44c0f1fbdd444f65b9160913c130bbfc"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "3fb4d5379a70cb48846e81f08106a4b0"
server_key_iv: "94ea620b1768e7266f6199ca0cd2842e"
server_key_version: "encryptionkey@0"
I20260806 09:00:38.357935 11825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:38.376518 11825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:38.380599 11825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:38.380867 11825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:38.381878 11825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:38.383298 11825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:38.383400 11825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:38.383514 11825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:38.383602 11825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 09:00:38.415707 11825 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:46475
I20260806 09:00:38.415767 12003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:46475 every 8 connection(s)
I20260806 09:00:38.417214 11825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:38.422225 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.420170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:45991)
0806 09:00:38.421021 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.421263 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":363,"threads_started":1}
W20260806 09:00:38.422890 12004 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:43991 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:43991: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
I20260806 09:00:38.423705 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.423371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:35387)
0806 09:00:38.423505 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.423600 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 09:00:38.424669 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.424306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:35541)
0806 09:00:38.424462 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.424538 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 09:00:38.424934 12004 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:00:38.426072 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 11825
I20260806 09:00:38.426642 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:45893
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:43991
--webserver_port=43137
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:38.767998 12008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:38.768311 12008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:38.768409 12008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:38.775906 12008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:38.776022 12008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:38.776137 12008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:00:38.786172 12008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:45893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=43137
--tserver_master_addrs=127.22.98.62:43991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:38.787499 12008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:38.788932 12008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:38.802821 12014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:38.802857 12015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:38.802913 12017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:38.803836 12008 server_base.cc:1034] running on GCE node
I20260806 09:00:38.804565 12008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:38.805564 12008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:38.806815 12008 hybrid_clock.cc:648] HybridClock initialized: now 1786006838806744 us; error 50 us; skew 500 ppm
I20260806 09:00:38.807183 12008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:38 dist-test-slave-jpj1 krb5kdc[9970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006838, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:38.810541 12008 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:38.812696 12008 webserver.cc:460] Webserver started at http://127.22.98.3:43137/ using document root <none> and password file <none>
I20260806 09:00:38.813700 12008 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:38.813874 12008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:38.822686 12008 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 09:00:38.826277 12025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:38.828145 12008 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:00:38.828347 12008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "b8285d05e89140fb9d9b1cbb2c4dc004"
format_stamp: "Formatted at 2026-08-06 09:00:32 on dist-test-slave-jpj1"
server_key: "0082fc114cce70f5871e9e44736ab002"
server_key_iv: "14e7425e86bb885bb1c572fa1fa804a4"
server_key_version: "encryptionkey@0"
I20260806 09:00:38.829834 12008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:38.871332 12008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:38.875298 12008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:38.875595 12008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:38.876617 12008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:38.877981 12008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:38.878086 12008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:38.878203 12008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:38.878283 12008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:38.910493 12008 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:45893
I20260806 09:00:38.910578 12181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:45893 every 8 connection(s)
I20260806 09:00:38.912047 12008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:38.916085 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 12008
I20260806 09:00:38.917161 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.915009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:52401)
0806 09:00:38.915822 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.916037 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":287,"threads_started":1}
W20260806 09:00:38.917948 12182 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:43991 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:43991: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
I20260806 09:00:38.918773 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.918454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:32825)
0806 09:00:38.918594 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.918666 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 09:00:38.919626 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.919266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:40783)
0806 09:00:38.919407 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.919481 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:00:38.919862 12182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:00:38.944095 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:38.943201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:52239)
0806 09:00:38.943778 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:38.943902 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":225,"threads_started":1}
I20260806 09:00:39.426397 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.425546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:55461)
0806 09:00:39.426034 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:39.426153 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":256,"threads_started":1}
I20260806 09:00:39.722517 12191 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:39.722887 12191 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:39.724742 12191 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
Aug 06 09:00:39 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006837, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
Aug 06 09:00:39 dist-test-slave-jpj1 krb5kdc[9970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006837, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:00:39.762286 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.725672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:49027 (local address 127.22.98.3:45893)
0806 09:00:39.737896 (+ 12224us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:39.737930 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:39.738742 (+ 812us) server_negotiation.cc:407] Connection header received
0806 09:00:39.740752 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:39.740761 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:39.741144 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:39.741461 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:39.743721 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.745021 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.746851 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.747479 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.747684 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:39.754977 (+ 7293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:39.754990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:39.755011 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:39.755068 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:39.758297 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759224 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:39.759230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:39.759237 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:39.759414 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759935 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:39.759939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:39.759941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:39.760498 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:39.760761 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:39.761572 (+ 811us) server_negotiation.cc:299] Negotiation successful
0806 09:00:39.761962 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11963,"thread_start_us":11823,"threads_started":1}
I20260806 09:00:39.762394 12193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.725244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45893 (local address 127.22.98.1:49027)
0806 09:00:39.738629 (+ 13385us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:39.738645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:39.738987 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:39.741636 (+ 2649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:39.741648 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:39.742654 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:39.743488 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:39.743512 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.745216 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.745220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:39.746715 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:39.746729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.747726 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.747737 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:39.748030 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:39.751957 (+ 3927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:39.751988 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:39.754607 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:39.758464 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:39.758473 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:39.758488 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:39.759068 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:39.759559 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:39.759569 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:39.759813 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:39.760652 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:39.760661 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:39.761015 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 09:00:39.761282 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 09:00:39.761910 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13255,"mutex_wait_us":193,"spinlock_wait_cycles":87808,"thread_start_us":13048,"threads_started":1}
I20260806 09:00:39.762439 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.725308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:56349 (local address 127.22.98.2:46475)
0806 09:00:39.738449 (+ 13141us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:39.738462 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:39.738516 (+ 54us) server_negotiation.cc:407] Connection header received
0806 09:00:39.740800 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:39.740810 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:39.741144 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:39.741475 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:39.743751 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.745037 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.746858 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.747487 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.747685 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:39.755421 (+ 7736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:39.755434 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:39.755452 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:39.755507 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:39.758424 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759221 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:39.759228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:39.759241 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:39.759410 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759951 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:39.759954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:39.759956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:39.760498 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:39.760676 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:39.761619 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 09:00:39.762039 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12959,"thread_start_us":12806,"threads_started":1}
I20260806 09:00:39.763041 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.725049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:46475 (local address 127.22.98.1:56349)
0806 09:00:39.738219 (+ 13170us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:39.738283 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:39.738976 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:39.741636 (+ 2660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:39.741648 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:39.742655 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:39.743492 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:39.743515 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.745211 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.745215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:39.746711 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:39.746729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:39.747763 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:39.747772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:39.748031 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:39.751959 (+ 3928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:39.751988 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:39.755066 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:39.758577 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:39.758581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:39.758584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:39.759072 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:39.759558 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:39.759564 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:39.759568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:39.759823 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:39.760653 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:39.760662 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:39.761019 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 09:00:39.761282 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 09:00:39.762755 (+ 1473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13017,"mutex_wait_us":1195,"spinlock_wait_cycles":33664,"thread_start_us":12873,"threads_started":1}
W20260806 09:00:39.766445 11662 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
W20260806 09:00:39.767431 11663 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:39.767580 11663 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f; no voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004
I20260806 09:00:39.768028 12191 raft_consensus.cc:2726] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:00:39.921343 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.920445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:60549)
0806 09:00:39.920925 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:39.921066 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":216,"threads_started":1}
I20260806 09:00:39.945480 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:39.944990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:52491)
0806 09:00:39.945181 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:39.945312 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 09:00:40.428067 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:40.427355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:50357)
0806 09:00:40.427816 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:40.427903 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":246,"threads_started":1}
I20260806 09:00:40.923168 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:40.922411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:34443)
0806 09:00:40.922905 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:40.923000 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":239,"threads_started":1}
I20260806 09:00:40.947342 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:40.946485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:34653)
0806 09:00:40.947051 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:40.947139 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":315,"threads_started":1}
I20260806 09:00:41.429841 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:41.429061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:51131)
0806 09:00:41.429535 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:41.429652 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":238,"threads_started":1}
I20260806 09:00:41.532079 12205 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:41.532277 12205 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:41.532922 12205 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
W20260806 09:00:41.534061 11662 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
W20260806 09:00:41.534274 11663 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:41.534435 11663 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f; no voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004
I20260806 09:00:41.534689 12205 raft_consensus.cc:2726] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:00:41.925117 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:41.924287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:51029)
0806 09:00:41.924842 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:41.924939 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
I20260806 09:00:41.949204 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:41.948343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:40975)
0806 09:00:41.948891 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:41.949007 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":269,"threads_started":1}
I20260806 09:00:42.431690 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:42.430847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:48285)
0806 09:00:42.431348 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:42.431439 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":248,"threads_started":1}
I20260806 09:00:42.927037 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:42.926226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:38331)
0806 09:00:42.926753 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:42.926849 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":252,"threads_started":1}
I20260806 09:00:42.951310 12211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:42.950411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:58985)
0806 09:00:42.950974 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:42.951064 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":288,"threads_started":1}
I20260806 09:00:43.328137 12213 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:43.328342 12213 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:43.328991 12213 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
W20260806 09:00:43.330087 11662 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
W20260806 09:00:43.330292 11663 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:43.330446 11663 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f; no voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004
I20260806 09:00:43.330705 12213 raft_consensus.cc:2726] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:00:43.433368 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:43.432598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:43555)
0806 09:00:43.433112 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:43.433196 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":260,"threads_started":1}
I20260806 09:00:43.928964 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:43.928163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:46523)
0806 09:00:43.928681 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:43.928769 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":264,"threads_started":1}
I20260806 09:00:43.953043 12219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:43.952339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:54349)
0806 09:00:43.952766 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:43.952870 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":211,"threads_started":1}
I20260806 09:00:44.435078 12221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:44.434254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:58067)
0806 09:00:44.434720 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:44.434812 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260806 09:00:44.930756 12222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:44.929929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:42863)
0806 09:00:44.930464 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:44.930555 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":292,"threads_started":1}
I20260806 09:00:44.954711 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:44.953921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:43781)
0806 09:00:44.954397 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:44.954494 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":246,"threads_started":1}
I20260806 09:00:45.436944 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:45.436035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:32849)
0806 09:00:45.436601 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:45.436692 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":279,"threads_started":1}
I20260806 09:00:45.932677 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:45.931812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:38339)
0806 09:00:45.932362 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:45.932451 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":250,"threads_started":1}
I20260806 09:00:45.956379 12228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:45.955651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:56697)
0806 09:00:45.956131 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:45.956219 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":234,"threads_started":1}
I20260806 09:00:46.421383 12229 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:46.421613 12229 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:46.422304 12229 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
W20260806 09:00:46.423287 11663 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
W20260806 09:00:46.423457 11662 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:46.423687 11662 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f; no voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004
I20260806 09:00:46.423955 12229 raft_consensus.cc:2726] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:00:46.438465 12230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:46.437779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:60029)
0806 09:00:46.438190 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:46.438277 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":207,"threads_started":1}
I20260806 09:00:46.934777 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:46.933890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:48687)
0806 09:00:46.934494 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:46.934584 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":250,"threads_started":1}
I20260806 09:00:46.958015 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:46.957232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:50625)
0806 09:00:46.957732 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:46.957823 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":261,"threads_started":1}
I20260806 09:00:47.440599 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:47.439596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:60061)
0806 09:00:47.440244 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:47.440417 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":367,"threads_started":1}
I20260806 09:00:47.936539 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:47.935730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:56649)
0806 09:00:47.936288 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:47.936384 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":268,"threads_started":1}
I20260806 09:00:47.959689 12239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:47.958978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:46993)
0806 09:00:47.959402 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:47.959485 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":221,"threads_started":1}
I20260806 09:00:48.442543 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:48.441762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:44449)
0806 09:00:48.442263 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:48.442357 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":258,"threads_started":1}
I20260806 09:00:48.557570 12245 raft_consensus.cc:482] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:48.557751 12245 raft_consensus.cc:504] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f82ee45bc2f4cd197b5baee374e824f" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 35047 } } peers { permanent_uuid: "44c0f1fbdd444f65b9160913c130bbfc" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 46475 } } peers { permanent_uuid: "b8285d05e89140fb9d9b1cbb2c4dc004" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45893 } }
I20260806 09:00:48.558398 12245 leader_election.cc:288] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475), b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893)
W20260806 09:00:48.559312 11662 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c0f1fbdd444f65b9160913c130bbfc (127.22.98.2:46475): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
W20260806 09:00:48.559481 11663 leader_election.cc:341] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8285d05e89140fb9d9b1cbb2c4dc004 (127.22.98.3:45893): Not found: Tablet not found: fbc7cd301895472f9783ba7e4d57f179
I20260806 09:00:48.559638 11663 leader_election.cc:302] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f82ee45bc2f4cd197b5baee374e824f; no voters: 44c0f1fbdd444f65b9160913c130bbfc, b8285d05e89140fb9d9b1cbb2c4dc004
I20260806 09:00:48.559885 12245 raft_consensus.cc:2726] T fbc7cd301895472f9783ba7e4d57f179 P 3f82ee45bc2f4cd197b5baee374e824f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:00:48.938194 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:48.937469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:55005)
0806 09:00:48.937907 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:48.937993 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":221,"threads_started":1}
I20260806 09:00:48.961606 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:48.960637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:59133)
0806 09:00:48.961175 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:48.961361 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":297,"threads_started":1}
I20260806 09:00:49.444469 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:49.443648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:33681)
0806 09:00:49.444170 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:49.444260 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":237,"threads_started":1}
I20260806 09:00:49.939898 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:49.939123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:54467)
0806 09:00:49.939642 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:49.939743 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":229,"threads_started":1}
I20260806 09:00:49.963310 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:49.962554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:57937)
0806 09:00:49.963034 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:49.963125 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":254,"threads_started":1}
I20260806 09:00:50.446442 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:50.445637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:35061)
0806 09:00:50.446145 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:50.446243 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":251,"threads_started":1}
I20260806 09:00:50.941452 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:50.940760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:48931)
0806 09:00:50.941199 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:50.941279 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":221,"threads_started":1}
I20260806 09:00:50.965116 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:50.964224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:50251)
0806 09:00:50.964794 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:50.964922 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":266,"threads_started":1}
I20260806 09:00:51.448500 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:51.447669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:57507)
0806 09:00:51.448203 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:51.448323 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":281,"threads_started":1}
I20260806 09:00:51.943168 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:51.942369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:41861)
0806 09:00:51.942892 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:51.942993 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":266,"threads_started":1}
I20260806 09:00:51.966998 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:51.966150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:47809)
0806 09:00:51.966691 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:51.966788 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
I20260806 09:00:52.450536 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:52.449670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:33091)
0806 09:00:52.450238 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:52.450333 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":282,"threads_started":1}
I20260806 09:00:52.945012 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:52.944207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:45053)
0806 09:00:52.944694 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:52.944788 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":249,"threads_started":1}
I20260806 09:00:52.969403 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:52.968503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:53467)
0806 09:00:52.969059 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:52.969158 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":287,"threads_started":1}
I20260806 09:00:53.452790 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:53.451919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.2:58277)
0806 09:00:53.452426 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:53.452567 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":229,"threads_started":1}
I20260806 09:00:53.927209 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 11591
I20260806 09:00:53.946141 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:53.928536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.22.98.1:35047)
0806 09:00:53.929203 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:53.929213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:53.929827 (+ 614us) server_negotiation.cc:407] Connection header received
0806 09:00:53.930030 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:53.930040 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:53.930307 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:53.930464 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:53.931985 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:53.932991 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:53.934325 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:53.934978 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:53.935106 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:53.940501 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:53.940515 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:53.940532 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:53.940570 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:53.943237 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:53.943983 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:53.943990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:53.943993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:53.944142 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:53.944534 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:53.944538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:53.944540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:53.944911 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:53.945222 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:53.945557 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 09:00:53.945822 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":252,"threads_started":1}
I20260806 09:00:53.947343 11813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:53.947458 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:53.946126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.3:49025)
0806 09:00:53.947186 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:53.947294 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":838,"threads_started":1}
I20260806 09:00:54.038255 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 11591
I20260806 09:00:54.038311 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:54.037249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:43991 (local address 127.22.98.1:34733)
0806 09:00:54.037806 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:54.037954 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:43991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":259,"threads_started":1}
I20260806 09:00:54.062196 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 11825
I20260806 09:00:54.076932 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:54.062949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.22.98.2:46475)
0806 09:00:54.063495 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:54.063500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:54.063517 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:00:54.063625 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:54.063672 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:54.063743 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:54.063847 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:54.065208 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:54.066118 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:54.067361 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:54.068072 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:54.068183 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:54.070883 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:54.070889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:54.070892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:54.070930 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:54.073899 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:54.074685 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:54.074691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:54.074693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:54.074858 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:54.075288 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:54.075293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:54.075295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:54.075677 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:54.075950 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:54.076331 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 09:00:54.076524 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":316,"threads_started":1}
I20260806 09:00:54.077761 11997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:54.160784 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 11825
I20260806 09:00:54.185017 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 12008
I20260806 09:00:54.198910 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:54.185891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.22.98.3:45893)
0806 09:00:54.186476 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:54.186484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:54.186503 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:00:54.186566 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:54.186571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:54.186639 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:54.186747 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:54.188260 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:54.189063 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:54.190269 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:54.190692 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:54.190821 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:54.193456 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:54.193461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:54.193464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:54.193498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:54.196176 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:54.196888 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:54.196893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:54.196895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:54.197045 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:54.197441 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:54.197444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:54.197447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:54.197712 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:54.197987 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:54.198290 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 09:00:54.198512 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":280,"threads_started":1}
I20260806 09:00:54.199764 12175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:54.282794 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 12008
2026-08-06T09:00:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24944 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:00:54 dist-test-slave-jpj1 krb5kdc[12280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:00:54 dist-test-slave-jpj1 krb5kdc[12280](info): set up 2 sockets
Aug 06 09:00:54 dist-test-slave-jpj1 krb5kdc[12280](info): commencing operation
krb5kdc: starting...
W20260806 09:00:55.478802 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:00:55 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:00:55.775681 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:56.125618 12295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:56.125941 12295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:56.126082 12295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:56.133626 12295 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:00:56.133767 12295 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:00:56.133836 12295 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:00:56.133910 12295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:00:56.133987 12295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:56.134094 12295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:56.134161 12295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:00:56.134238 12295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:00:56.144459 12295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:56.145797 12295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:56.147100 12295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:56.159217 12302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:56.159273 12306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:56.159252 12303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:56.159888 12295 server_base.cc:1034] running on GCE node
I20260806 09:00:56.160792 12295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:56.162278 12295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:56.163555 12295 hybrid_clock.cc:648] HybridClock initialized: now 1786006856163466 us; error 36 us; skew 500 ppm
I20260806 09:00:56.163913 12295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:56 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006856, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:56.167217 12295 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:00:56.169034 12295 webserver.cc:460] Webserver started at http://127.22.98.62:39311/ using document root <none> and password file <none>
I20260806 09:00:56.169845 12295 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:56.169947 12295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:56.170300 12295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:56.173558 12295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "e752207e9bae4fbda2fb702e1d5a909d"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "e8cb8876af9512f36d0d5106c2043e1d"
server_key_iv: "64859c0f0dbb112f541b5f8402fa3bdf"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.174420 12295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "e752207e9bae4fbda2fb702e1d5a909d"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "e8cb8876af9512f36d0d5106c2043e1d"
server_key_iv: "64859c0f0dbb112f541b5f8402fa3bdf"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.180135 12295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:00:56.183925 12314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:56.185391 12295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:00:56.185556 12295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "e752207e9bae4fbda2fb702e1d5a909d"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "e8cb8876af9512f36d0d5106c2043e1d"
server_key_iv: "64859c0f0dbb112f541b5f8402fa3bdf"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.185779 12295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:56.201164 12295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:56.205582 12295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:56.205849 12295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:56.224382 12295 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34307
I20260806 09:00:56.224406 12386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34307 every 8 connection(s)
I20260806 09:00:56.225800 12295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:00:56.229647 12390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:56.235864 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 12295
I20260806 09:00:56.236090 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:00:56.236914 22920 external_mini_cluster.cc:1428] Setting key c2e1a25c85bf38d947277b2ce82e1437
I20260806 09:00:56.241343 12390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap starting.
I20260806 09:00:56.245471 12390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:56.246750 12390 log.cc:824] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:56.250576 12390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: No bootstrap required, opened a new log
Aug 06 09:00:56 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:56.259478 12390 raft_consensus.cc:348] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:00:56.259987 12390 raft_consensus.cc:374] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:56.260092 12390 raft_consensus.cc:729] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Initialized, Role: FOLLOWER
I20260806 09:00:56.260761 12390 consensus_queue.cc:260] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:00:56.261015 12390 raft_consensus.cc:388] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:00:56.261197 12390 raft_consensus.cc:482] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:00:56.261409 12390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:56.262568 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:56.239178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.22.98.62:34307)
0806 09:00:56.239989 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:56.240002 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:56.240045 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:00:56.242102 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:56.242131 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:56.242565 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:56.242929 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:56.244906 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.246418 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.248319 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.249104 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.249269 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:56.254541 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:56.254555 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:56.254573 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:56.254627 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:56.257995 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:56.258808 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:56.258816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:56.258823 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:56.259014 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:56.259415 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:56.259419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:56.259421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:56.260178 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:56.260489 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:56.260978 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 09:00:56.261457 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":379,"threads_started":1}
I20260806 09:00:56.264025 12390 raft_consensus.cc:504] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:00:56.264513 12390 leader_election.cc:302] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e752207e9bae4fbda2fb702e1d5a909d; no voters:
I20260806 09:00:56.265177 12390 leader_election.cc:288] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:00:56.265321 12397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:56.266011 12397 raft_consensus.cc:686] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 LEADER]: Becoming Leader. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Running, Role: LEADER
I20260806 09:00:56.266631 12397 consensus_queue.cc:237] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:00:56.267613 12390 sys_catalog.cc:564] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:00:56.269919 12399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:00:56.270520 12399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:56.270061 12400 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e752207e9bae4fbda2fb702e1d5a909d. Latest consensus state: current_term: 1 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:00:56.276739 12400 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:00:56.279320 12408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:00:56.284454 12408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:00:56.298588 12408 catalog_manager.cc:1347] Generated new cluster ID: de7c52ce20084fbbb9d91682ac5e47dd
I20260806 09:00:56.298758 12408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:00:56.312431 12408 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:00:56.313776 12408 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:00:56.323307 12408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Generated new TSK 0
I20260806 09:00:56.324461 12408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:00:56.398974 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:56.748772 12426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:56.749054 12426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:56.749179 12426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:56.756609 12426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:56.756735 12426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:56.756911 12426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:00:56.767802 12426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:56.769280 12426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:56.770752 12426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:56.783944 12435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:56.783985 12436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:56.784014 12438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:56.785202 12426 server_base.cc:1034] running on GCE node
I20260806 09:00:56.785892 12426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:56.786921 12426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:56.788156 12426 hybrid_clock.cc:648] HybridClock initialized: now 1786006856788098 us; error 52 us; skew 500 ppm
I20260806 09:00:56.788530 12426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:56 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006856, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:56.791832 12426 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:00:56.793308 12426 webserver.cc:460] Webserver started at http://127.22.98.1:35779/ using document root <none> and password file <none>
I20260806 09:00:56.793994 12426 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:56.794114 12426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:56.794428 12426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:56.797588 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "319226ebee8b40698e938768c4bbb083"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "908316f81f6a54f91b55f5ddeeffc01b"
server_key_iv: "e6bfe209e7d0b61815f0f2c186b52a6a"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.798359 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "319226ebee8b40698e938768c4bbb083"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "908316f81f6a54f91b55f5ddeeffc01b"
server_key_iv: "e6bfe209e7d0b61815f0f2c186b52a6a"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.803658 12426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260806 09:00:56.807281 12445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:56.808895 12426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:00:56.809077 12426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "319226ebee8b40698e938768c4bbb083"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "908316f81f6a54f91b55f5ddeeffc01b"
server_key_iv: "e6bfe209e7d0b61815f0f2c186b52a6a"
server_key_version: "encryptionkey@0"
I20260806 09:00:56.809284 12426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:56.845196 12426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:56.849912 12426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:56.850204 12426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:56.851222 12426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:56.852633 12426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:56.852725 12426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:56.852849 12426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:56.852902 12426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:56.883862 12426 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:33535
I20260806 09:00:56.883934 12609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:33535 every 8 connection(s)
I20260806 09:00:56.885267 12426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:00:56.890087 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 12426
I20260806 09:00:56.890300 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:00:56.890980 22920 external_mini_cluster.cc:1428] Setting key baa93cd235407ed3317fdff7c4d5ea31
Aug 06 09:00:56 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006856, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:56.906392 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:56.887886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:33151 (local address 127.22.98.62:34307)
0806 09:00:56.888436 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:56.888441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:56.889557 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 09:00:56.891658 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:56.891663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:56.891744 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:56.891854 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:56.893764 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.894762 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.896433 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.896993 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.897120 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:56.900555 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:56.900561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:56.900566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:56.900605 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:56.903419 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:56.904165 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:56.904170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:56.904174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:56.904314 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:56.904692 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:56.904696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:56.904699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:56.904994 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:56.905104 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:56.905962 (+ 858us) server_negotiation.cc:299] Negotiation successful
0806 09:00:56.906135 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":307,"threads_started":1}
I20260806 09:00:56.907827 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:56.888574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:33151)
0806 09:00:56.889379 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:56.889432 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:56.891333 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:56.892101 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:56.892113 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:56.892888 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:56.893578 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:56.893597 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.894956 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.894961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:56.896292 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:56.896305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:56.897230 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:56.897236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:56.897437 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:56.898126 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:56.898150 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:56.900331 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:56.903589 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:56.903596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:56.903609 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:56.904036 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:56.904404 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:56.904407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:56.904409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:56.904588 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:56.905113 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:56.905121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:56.905555 (+ 434us) client_negotiation.cc:769] Sending connection context
0806 09:00:56.905861 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 09:00:56.906420 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":307,"threads_started":1}
I20260806 09:00:56.909410 12610 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:00:56.909721 12610 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:56.910444 12610 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:00:56.912572 12341 ts_manager.cc:194] Registered new tserver with Master: 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535)
I20260806 09:00:56.914016 12341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:33151
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 09:00:56.956769 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:57.307153 12617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:57.307451 12617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:57.307616 12617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:57.315078 12617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:57.315207 12617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:57.315359 12617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:00:57.325470 12617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:57.326932 12617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:57.328461 12617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:57.341821 12623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:57.341784 12624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:57.341853 12628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:57.342857 12617 server_base.cc:1034] running on GCE node
I20260806 09:00:57.343417 12617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:57.344547 12617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:57.345831 12617 hybrid_clock.cc:648] HybridClock initialized: now 1786006857345701 us; error 58 us; skew 500 ppm
I20260806 09:00:57.346153 12617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:57 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006857, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:57.349421 12617 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:00:57.350962 12617 webserver.cc:460] Webserver started at http://127.22.98.2:33165/ using document root <none> and password file <none>
I20260806 09:00:57.351701 12617 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:57.351843 12617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:57.352137 12617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:57.355087 12617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "420486162cda414787cbce4df4dcc874"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "c05e3c35eba3cc43c69b330692238a5e"
server_key_iv: "c87ba7dfb89174e90278485fa02dbb40"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.355899 12617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "420486162cda414787cbce4df4dcc874"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "c05e3c35eba3cc43c69b330692238a5e"
server_key_iv: "c87ba7dfb89174e90278485fa02dbb40"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.361076 12617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:00:57.364588 12637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.366014 12617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:00:57.366194 12617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "420486162cda414787cbce4df4dcc874"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "c05e3c35eba3cc43c69b330692238a5e"
server_key_iv: "c87ba7dfb89174e90278485fa02dbb40"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.366402 12617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:57.396888 12617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:57.401371 12617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:57.401681 12617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:57.404192 12617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:57.406090 12617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:57.406190 12617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.406322 12617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:57.406383 12617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.440234 12617 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39195
I20260806 09:00:57.440307 12807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39195 every 8 connection(s)
I20260806 09:00:57.441764 12617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:00:57.446513 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 12617
I20260806 09:00:57.446727 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:00:57.447309 22920 external_mini_cluster.cc:1428] Setting key ea74161fc189e669ecb1192cb809a074
Aug 06 09:00:57 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006857, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:57.465355 12810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:57.444488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:56735 (local address 127.22.98.62:34307)
0806 09:00:57.444953 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:57.444957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:57.445923 (+ 966us) server_negotiation.cc:407] Connection header received
0806 09:00:57.448611 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:57.448618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:57.448721 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:57.448833 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:57.450972 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.451830 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.454049 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.454527 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.454642 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:57.459072 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:57.459079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:57.459082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:57.459121 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:57.461835 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:57.462736 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:57.462742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:57.462746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:57.462939 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:57.463370 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:57.463374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:57.463377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:57.463803 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:57.463964 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:57.464727 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 09:00:57.464979 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260806 09:00:57.466277 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:57.444946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:56735)
0806 09:00:57.445727 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:57.445776 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:57.448254 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:57.449159 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:57.449168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:57.449829 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:57.450748 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:57.450790 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.452023 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.452029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:57.453868 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:57.453886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.454796 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.454804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:57.455113 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:57.455961 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:57.455992 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:57.458766 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:57.462000 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:57.462009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:57.462054 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:57.462586 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:57.463056 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:57.463059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:57.463062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:57.463264 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:57.463966 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:57.463973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:57.464338 (+ 365us) client_negotiation.cc:769] Sending connection context
0806 09:00:57.464590 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 09:00:57.465072 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":337,"threads_started":1}
I20260806 09:00:57.468072 12808 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:00:57.468427 12808 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:57.469252 12808 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:00:57.470999 12341 ts_manager.cc:194] Registered new tserver with Master: 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195)
I20260806 09:00:57.472080 12341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:56735
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:00:57.518918 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:00:57.860266 12817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:00:57.860549 12817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:00:57.860687 12817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:00:57.868037 12817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:00:57.868163 12817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:00:57.868320 12817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:00:57.878034 12817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:00:57.879328 12817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:00:57.880728 12817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:00:57.894270 12826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:57.894297 12831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:00:57.894237 12827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:00:57.900449 12817 server_base.cc:1034] running on GCE node
I20260806 09:00:57.901190 12817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:00:57.902213 12817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:00:57.903420 12817 hybrid_clock.cc:648] HybridClock initialized: now 1786006857903361 us; error 46 us; skew 500 ppm
I20260806 09:00:57.903791 12817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:00:57 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006857, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:00:57.907003 12817 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:00:57.908743 12817 webserver.cc:460] Webserver started at http://127.22.98.3:41503/ using document root <none> and password file <none>
I20260806 09:00:57.909487 12817 fs_manager.cc:359] Metadata directory not provided
I20260806 09:00:57.909621 12817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:00:57.909935 12817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:00:57.913008 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "3bd50ea1b377fa360c4f6fd33755b1e8"
server_key_iv: "1f690e59a65daa3fbaa05b9ffbdfae5b"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.913774 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "3bd50ea1b377fa360c4f6fd33755b1e8"
server_key_iv: "1f690e59a65daa3fbaa05b9ffbdfae5b"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.917029 12610 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
I20260806 09:00:57.919188 12817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:00:57.922756 12845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.924099 12817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:00:57.924249 12817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "3bd50ea1b377fa360c4f6fd33755b1e8"
server_key_iv: "1f690e59a65daa3fbaa05b9ffbdfae5b"
server_key_version: "encryptionkey@0"
I20260806 09:00:57.924435 12817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:00:57.945706 12817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:00:57.950080 12817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:00:57.950364 12817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:00:57.951346 12817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:00:57.952782 12817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:00:57.952867 12817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.952991 12817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:00:57.953047 12817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:00:57.984690 12817 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:33687
I20260806 09:00:57.984829 13016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:33687 every 8 connection(s)
I20260806 09:00:57.986042 12817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:00:57.988600 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 12817
I20260806 09:00:57.988783 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:00:57.989411 22920 external_mini_cluster.cc:1428] Setting key 11ff248b995dd01c266545f91d7f9bc2
Aug 06 09:00:57 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006857, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:00:58.006593 13020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:57.988836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:59801 (local address 127.22.98.62:34307)
0806 09:00:57.989372 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:57.989378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:57.990476 (+ 1098us) server_negotiation.cc:407] Connection header received
0806 09:00:57.992572 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:57.992577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:57.992676 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:57.992774 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:57.994368 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.995187 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.996778 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.997171 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.997285 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.000725 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:58.000731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:58.000734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:58.000767 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:58.003192 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.004081 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.004086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.004088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.004216 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.004709 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.004713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.004715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.005047 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:58.005262 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.005960 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.006208 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":224,"threads_started":1}
I20260806 09:00:58.007627 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:57.989373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:59801)
0806 09:00:57.990302 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:57.990360 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:57.992308 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:57.993051 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:57.993060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:57.993594 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:00:57.994191 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:57.994213 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.995341 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.995345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:57.996631 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:57.996646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:57.997398 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:57.997406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:57.997619 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:57.998266 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:00:57.998303 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:00:58.000507 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:00:58.003333 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:58.003342 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:58.003361 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:58.003978 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:58.004323 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:00:58.004328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:00:58.004332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:00:58.004587 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:00:58.005183 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:00:58.005191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:00:58.005569 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.005823 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.006381 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":383,"threads_started":1}
I20260806 09:00:58.009155 13018 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:00:58.009445 13018 heartbeater.cc:464] Registering TS with master...
I20260806 09:00:58.010236 13018 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:00:58.011904 12341 ts_manager.cc:194] Registered new tserver with Master: cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
I20260806 09:00:58.012822 12341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:59801
I20260806 09:00:58.018256 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:00:58.036474 13020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.023087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.22.98.62:34307)
0806 09:00:58.023256 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.023261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.023806 (+ 545us) server_negotiation.cc:407] Connection header received
0806 09:00:58.023927 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.023932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.024020 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.024112 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:58.025827 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.026598 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.028308 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.028750 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.028896 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.031241 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:58.031246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:58.031250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:58.031286 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:58.033911 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.034569 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.034611 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.034613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.034757 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.035109 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.035113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.035115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.035462 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:58.035749 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.036032 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.036227 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:00:58.040589 12341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:00:58.047379 12341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:00:58.082581 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.060488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:43268)
0806 09:00:58.073102 (+ 12614us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:58.073120 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:58.073233 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:58.074551 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:58.074559 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:58.074610 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:58.075107 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.075130 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.077286 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.077291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.079589 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.079603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.081501 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.081509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.081703 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.081764 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.081909 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.082146 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12476,"thread_start_us":12340,"threads_started":1}
I20260806 09:00:58.083242 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.061678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:33687 (local address 127.0.0.1:46712)
0806 09:00:58.075837 (+ 14159us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:58.075855 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:58.075982 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:58.076588 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:58.076593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:58.076619 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:58.076981 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.076991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.078896 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.078901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.080654 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.080667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.082455 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.082461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.082655 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.082688 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.082781 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.082903 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13555,"thread_start_us":13771,"threads_started":1}
I20260806 09:00:58.083434 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.060520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.22.98.2:39195)
0806 09:00:58.074327 (+ 13807us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.074336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.074365 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:00:58.074477 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.074485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.074721 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.074934 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:58.075944 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.077245 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.080005 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.081415 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.081529 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.081724 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.082897 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.083082 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13653,"thread_start_us":13515,"threads_started":1}
I20260806 09:00:58.083648 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.060747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.22.98.1:33535)
0806 09:00:58.073803 (+ 13056us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.073813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.073845 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:00:58.073977 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.073987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.074266 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.075363 (+ 1097us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:58.075655 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.077136 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.079746 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.081276 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.081429 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.083033 (+ 1604us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.083111 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.083274 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12887,"thread_start_us":12707,"threads_started":1}
I20260806 09:00:58.084175 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.062345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.22.98.3:33687)
0806 09:00:58.074889 (+ 12544us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.074897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.075975 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 09:00:58.076137 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.076146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.076386 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.076570 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:58.077166 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.078721 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.080818 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.081960 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.082076 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.082306 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.083558 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.083815 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12383,"thread_start_us":12220,"threads_started":1}
I20260806 09:00:58.082723 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.060467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39195 (local address 127.0.0.1:41234)
0806 09:00:58.072699 (+ 12232us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:58.072768 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:58.073061 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:58.075203 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:58.075208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:58.075241 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:58.075729 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.075744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.077400 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.077405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.079827 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.079844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.081889 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.081896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.082078 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.082113 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.082202 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.082355 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11852,"thread_start_us":13251,"threads_started":1}
I20260806 09:00:58.088441 12922 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (DEFAULT_TABLE table=test-table [id=ece5255b9c564213a2f1dc0092c7600c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:58.088438 12706 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (DEFAULT_TABLE table=test-table [id=ece5255b9c564213a2f1dc0092c7600c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:58.088438 12514 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (DEFAULT_TABLE table=test-table [id=ece5255b9c564213a2f1dc0092c7600c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:00:58.089946 12922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:58.089965 12706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:58.089964 12514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:00:58.102495 13041 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Bootstrap starting.
I20260806 09:00:58.105283 13042 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Bootstrap starting.
I20260806 09:00:58.106870 13046 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Bootstrap starting.
I20260806 09:00:58.108206 13041 tablet_bootstrap.cc:654] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:58.108953 13042 tablet_bootstrap.cc:654] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:58.109593 13041 log.cc:824] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:58.109790 13046 tablet_bootstrap.cc:654] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Neither blocks nor log segments found. Creating new log.
I20260806 09:00:58.110248 13042 log.cc:824] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:58.110935 13046 log.cc:824] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Log is configured to *not* fsync() on all Append() calls
I20260806 09:00:58.112972 13041 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: No bootstrap required, opened a new log
I20260806 09:00:58.113092 13042 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: No bootstrap required, opened a new log
I20260806 09:00:58.113286 13041 ts_tablet_manager.cc:1397] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 09:00:58.113379 13042 ts_tablet_manager.cc:1397] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 09:00:58.113963 13046 tablet_bootstrap.cc:492] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: No bootstrap required, opened a new log
I20260806 09:00:58.114212 13046 ts_tablet_manager.cc:1397] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 09:00:58.121196 13042 raft_consensus.cc:348] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.121604 13042 raft_consensus.cc:374] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:58.121711 13042 raft_consensus.cc:729] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420486162cda414787cbce4df4dcc874, State: Initialized, Role: FOLLOWER
I20260806 09:00:58.122370 13042 consensus_queue.cc:260] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.123624 12808 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
I20260806 09:00:58.124405 13042 ts_tablet_manager.cc:1428] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260806 09:00:58.124356 13041 raft_consensus.cc:348] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.124845 13041 raft_consensus.cc:374] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:58.124943 13041 raft_consensus.cc:729] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec819cf4a8a4b768b3cbf23f8ff83a8, State: Initialized, Role: FOLLOWER
I20260806 09:00:58.124960 13046 raft_consensus.cc:348] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.125383 13046 raft_consensus.cc:374] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:00:58.125478 13046 raft_consensus.cc:729] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319226ebee8b40698e938768c4bbb083, State: Initialized, Role: FOLLOWER
I20260806 09:00:58.125636 13041 consensus_queue.cc:260] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.126082 13046 consensus_queue.cc:260] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.129240 13046 ts_tablet_manager.cc:1428] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260806 09:00:58.131415 13018 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
I20260806 09:00:58.133414 13041 ts_tablet_manager.cc:1428] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Time spent starting tablet: real 0.020s user 0.007s sys 0.011s
W20260806 09:00:58.137701 12612 tablet.cc:2369] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:00:58.193893 12809 tablet.cc:2369] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:58.229444 13054 raft_consensus.cc:482] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:00:58.229794 13054 raft_consensus.cc:504] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.231571 13054 leader_election.cc:288] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
W20260806 09:00:58.237659 13019 tablet.cc:2369] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:00:58.239626 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.231981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:55063 (local address 127.22.98.3:33687)
0806 09:00:58.232194 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.232200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.233456 (+ 1256us) server_negotiation.cc:407] Connection header received
0806 09:00:58.233610 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.233615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.233692 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.233784 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:58.234413 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.235470 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.237302 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.238590 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.238722 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.238829 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.239235 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.239361 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:00:58.239799 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.232004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:33687 (local address 127.22.98.2:55063)
0806 09:00:58.233360 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:58.233375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:58.233487 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:58.233806 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:58.233811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:58.233842 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:58.234250 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.234261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.235623 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.235629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.237143 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.237157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.238956 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.238961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.239077 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.239102 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.239185 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.239318 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1244,"mutex_wait_us":2,"thread_start_us":674,"threads_started":1}
I20260806 09:00:58.240402 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.232445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:32821 (local address 127.22.98.1:33535)
0806 09:00:58.232679 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.232685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.233123 (+ 438us) server_negotiation.cc:407] Connection header received
0806 09:00:58.233422 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.233429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.233506 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.233594 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:00:58.234210 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.235290 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.237973 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.239200 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.239305 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.239406 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.240021 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.240173 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:00:58.240820 12951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa33b1b8f9c47c38bccffa2bd02aa6f" candidate_uuid: "420486162cda414787cbce4df4dcc874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" is_pre_election: true
I20260806 09:00:58.241068 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.231809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.22.98.2:32821)
0806 09:00:58.233001 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 09:00:58.233023 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:00:58.233257 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:00:58.233626 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:00:58.233630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:00:58.233652 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:00:58.234064 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.234076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.235434 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.235439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.237785 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:00:58.237800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.239636 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.239642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:00:58.239840 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.239872 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:00:58.240580 (+ 708us) client_negotiation.cc:240] Negotiation successful
0806 09:00:58.240724 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":94,"thread_start_us":1233,"threads_started":1}
I20260806 09:00:58.241343 12951 raft_consensus.cc:2454] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 420486162cda414787cbce4df4dcc874 in term 0.
I20260806 09:00:58.242122 12641 leader_election.cc:302] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8; no voters:
I20260806 09:00:58.242013 12548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa33b1b8f9c47c38bccffa2bd02aa6f" candidate_uuid: "420486162cda414787cbce4df4dcc874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "319226ebee8b40698e938768c4bbb083" is_pre_election: true
I20260806 09:00:58.242558 13054 raft_consensus.cc:2781] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:00:58.242560 12548 raft_consensus.cc:2454] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 420486162cda414787cbce4df4dcc874 in term 0.
I20260806 09:00:58.242693 13054 raft_consensus.cc:482] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:00:58.242802 13054 raft_consensus.cc:3037] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:58.245105 13054 raft_consensus.cc:504] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.245738 13054 leader_election.cc:288] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [CANDIDATE]: Term 1 election: Requested vote from peers 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
I20260806 09:00:58.246150 12548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa33b1b8f9c47c38bccffa2bd02aa6f" candidate_uuid: "420486162cda414787cbce4df4dcc874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "319226ebee8b40698e938768c4bbb083"
I20260806 09:00:58.246236 12951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa33b1b8f9c47c38bccffa2bd02aa6f" candidate_uuid: "420486162cda414787cbce4df4dcc874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
I20260806 09:00:58.246331 12548 raft_consensus.cc:3037] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:58.246459 12951 raft_consensus.cc:3037] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:00:58.248651 12951 raft_consensus.cc:2454] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 420486162cda414787cbce4df4dcc874 in term 1.
I20260806 09:00:58.248664 12548 raft_consensus.cc:2454] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 420486162cda414787cbce4df4dcc874 in term 1.
I20260806 09:00:58.249039 12641 leader_election.cc:302] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8; no voters:
I20260806 09:00:58.249349 13054 raft_consensus.cc:2781] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:00:58.249814 13054 raft_consensus.cc:686] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 1 LEADER]: Becoming Leader. State: Replica: 420486162cda414787cbce4df4dcc874, State: Running, Role: LEADER
I20260806 09:00:58.250298 13054 consensus_queue.cc:237] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:00:58.255368 12341 catalog_manager.cc:5495] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 reported cstate change: term changed from 0 to 1, leader changed from <none> to 420486162cda414787cbce4df4dcc874 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "420486162cda414787cbce4df4dcc874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } health_report { overall_health: UNKNOWN } } }
I20260806 09:00:58.279888 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.274392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.22.98.2:39195)
0806 09:00:58.274666 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.274672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.274693 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:00:58.274828 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.274832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.274914 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.275047 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:00:58.275460 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.276251 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.277655 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.278052 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.278166 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.278708 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:00:58.279250 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:00:58.279400 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.279503 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.279619 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:00:58.288278 12951 raft_consensus.cc:1261] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Refusing update from remote peer 420486162cda414787cbce4df4dcc874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:58.288290 12548 raft_consensus.cc:1261] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Refusing update from remote peer 420486162cda414787cbce4df4dcc874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:00:58.289291 13060 consensus_queue.cc:1035] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:58.289778 13054 consensus_queue.cc:1035] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:00:58.302342 13067 mvcc.cc:205] Tried to move back new op lower bound from 7315484091538235392 to 7315484091396943872. Current Snapshot: MvccSnapshot[applied={T|T < 7315484091538235392}]
I20260806 09:00:58.304101 13065 mvcc.cc:205] Tried to move back new op lower bound from 7315484091538235392 to 7315484091396943872. Current Snapshot: MvccSnapshot[applied={T|T < 7315484091538235392 or (T in {7315484091538235392})}]
I20260806 09:00:58.315443 12340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:58.315842 12340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:00:58.319782 12340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Sending DeleteTablet for 3 replicas of tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f
I20260806 09:00:58.320907 12706 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:58 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:41234
I20260806 09:00:58.321076 12514 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:58 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:43268
I20260806 09:00:58.321437 12922 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:00:58 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:46712
I20260806 09:00:58.321811 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 12426
I20260806 09:00:58.332392 13081 tablet_replica.cc:330] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: stopping tablet replica
I20260806 09:00:58.333035 13081 raft_consensus.cc:2229] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:58.333709 13081 raft_consensus.cc:2258] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:58.334597 13079 tablet_replica.cc:330] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: stopping tablet replica
I20260806 09:00:58.334967 13079 raft_consensus.cc:2229] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:00:58.335517 13079 raft_consensus.cc:2258] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:00:58.337916 13079 ts_tablet_manager.cc:1905] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:58.340070 13080 tablet_replica.cc:330] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: stopping tablet replica
I20260806 09:00:58.340857 13081 ts_tablet_manager.cc:1905] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:58.346174 13079 ts_tablet_manager.cc:1918] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:58.346446 13079 log.cc:1196] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/7fa33b1b8f9c47c38bccffa2bd02aa6f
I20260806 09:00:58.347126 13079 ts_tablet_manager.cc:1939] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 420486162cda414787cbce4df4dcc874: Deleting consensus metadata
I20260806 09:00:58.347663 13080 raft_consensus.cc:2229] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:00:58.347790 13081 ts_tablet_manager.cc:1918] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:58.347980 13081 log.cc:1196] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/7fa33b1b8f9c47c38bccffa2bd02aa6f
I20260806 09:00:58.348306 13080 raft_consensus.cc:2258] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 09:00:58 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:00:58.348621 13081 ts_tablet_manager.cc:1939] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting consensus metadata
I20260806 09:00:58.350039 12316 catalog_manager.cc:4841] TS 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (table test-table [id=ece5255b9c564213a2f1dc0092c7600c]) successfully deleted
I20260806 09:00:58.350486 13080 ts_tablet_manager.cc:1905] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:00:58.350536 12317 catalog_manager.cc:4841] TS cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (table test-table [id=ece5255b9c564213a2f1dc0092c7600c]) successfully deleted
I20260806 09:00:58.356230 13080 ts_tablet_manager.cc:1918] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:00:58.356417 13080 log.cc:1196] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/7fa33b1b8f9c47c38bccffa2bd02aa6f
I20260806 09:00:58.356901 13080 ts_tablet_manager.cc:1939] T 7fa33b1b8f9c47c38bccffa2bd02aa6f P 319226ebee8b40698e938768c4bbb083: Deleting consensus metadata
I20260806 09:00:58.358779 12318 catalog_manager.cc:4841] TS 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535): tablet 7fa33b1b8f9c47c38bccffa2bd02aa6f (table test-table [id=ece5255b9c564213a2f1dc0092c7600c]) successfully deleted
I20260806 09:00:58.358860 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.323070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.22.98.1:33535)
0806 09:00:58.323268 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.323275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.335634 (+ 12359us) server_negotiation.cc:407] Connection header received
0806 09:00:58.336157 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.336164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.336304 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.336392 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:58.340078 (+ 3686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.341232 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.343301 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.344004 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.344103 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.350596 (+ 6493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:58.350613 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:58.350632 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:58.350684 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:58.354760 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.355835 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.355846 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.355850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.356039 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.356628 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.356635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.356638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.357161 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:58.357501 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.357884 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.358151 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:00:58.359524 12602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:58.454370 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 12426
I20260806 09:00:58.480111 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 12617
Aug 06 09:00:58 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:00:58.497546 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.481299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.22.98.2:39195)
0806 09:00:58.481476 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.481481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.481498 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:00:58.481557 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.481561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.481621 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.481737 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:58.483189 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.484065 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.485460 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.485919 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.486031 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.490917 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:58.490932 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:58.490950 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:58.490996 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:58.494192 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.494943 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.494952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.494954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.495114 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.495856 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.495861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.495863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.496305 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:58.496748 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.496939 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.497174 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:00:58.498494 12801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:58.613595 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 12617
I20260806 09:00:58.638048 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 12817
Aug 06 09:00:58 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:00:58.653482 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:00:58.638852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.22.98.3:33687)
0806 09:00:58.639035 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:00:58.639041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:00:58.639095 (+ 54us) server_negotiation.cc:407] Connection header received
0806 09:00:58.639281 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:00:58.639286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:00:58.639372 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:00:58.639458 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:00:58.640920 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.641627 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.642953 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:00:58.643353 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:00:58.643461 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:00:58.647704 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:00:58.647719 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:00:58.647738 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:00:58.647774 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:00:58.650485 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.651240 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.651249 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.651252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.651415 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:00:58.651848 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:00:58.651854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:00:58.651856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:00:58.652256 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:00:58.652562 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 09:00:58.652896 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 09:00:58.653125 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:00:58.654189 13010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:58.743148 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 12817
I20260806 09:00:58.763748 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 12295
I20260806 09:00:58.764932 12380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:00:58.877199 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 12295
I20260806 09:01:18.904564 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--webserver_interface=127.22.98.62
--webserver_port=39311
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:01:19.272899 13091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:19.273253 13091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:19.273398 13091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:19.280905 13091 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:01:19.281023 13091 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:01:19.281082 13091 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:01:19.281158 13091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:01:19.281235 13091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:19.281334 13091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:19.281400 13091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:01:19.281476 13091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:01:19.292256 13091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=39311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:19.293637 13091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:19.295113 13091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:19.321908 13100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:19.321985 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:19.322299 13091 server_base.cc:1034] running on GCE node
W20260806 09:01:19.322023 13098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:19.323618 13091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:19.325367 13091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:19.326678 13091 hybrid_clock.cc:648] HybridClock initialized: now 1786006879326548 us; error 56 us; skew 500 ppm
I20260806 09:01:19.327286 13091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:19 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:19.331130 13091 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:01:19.333004 13091 webserver.cc:460] Webserver started at http://127.22.98.62:39311/ using document root <none> and password file <none>
I20260806 09:01:19.334015 13091 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:19.334138 13091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:19.341240 13091 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 09:01:19.344810 13112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:19.346686 13091 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 09:01:19.346905 13091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "e752207e9bae4fbda2fb702e1d5a909d"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "e8cb8876af9512f36d0d5106c2043e1d"
server_key_iv: "64859c0f0dbb112f541b5f8402fa3bdf"
server_key_version: "encryptionkey@0"
I20260806 09:01:19.348122 13091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:19.363016 13091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:19.367657 13091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:19.367970 13091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:19.389070 13091 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34307
I20260806 09:01:19.389099 13194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34307 every 8 connection(s)
I20260806 09:01:19.390715 13091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:01:19.397157 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13091
I20260806 09:01:19.398223 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:33535
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=35779
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:01:19.398800 13195 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:01:19.403666 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap starting.
I20260806 09:01:19.426664 13195 log.cc:824] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:19.440646 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:01:19.441215 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap complete.
I20260806 09:01:19.450693 13195 raft_consensus.cc:348] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:19.451617 13195 raft_consensus.cc:729] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Initialized, Role: FOLLOWER
I20260806 09:01:19.452301 13195 consensus_queue.cc:260] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:19.452560 13195 raft_consensus.cc:388] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:01:19.452692 13195 raft_consensus.cc:482] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:01:19.452863 13195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:01:19.457065 13195 raft_consensus.cc:504] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:19.457685 13195 leader_election.cc:302] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e752207e9bae4fbda2fb702e1d5a909d; no voters:
I20260806 09:01:19.458386 13195 leader_election.cc:288] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:01:19.458552 13199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:01:19.459753 13199 raft_consensus.cc:686] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 LEADER]: Becoming Leader. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Running, Role: LEADER
I20260806 09:01:19.460516 13199 consensus_queue.cc:237] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:19.461174 13195 sys_catalog.cc:564] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:01:19.463399 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e752207e9bae4fbda2fb702e1d5a909d. Latest consensus state: current_term: 2 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:01:19.463475 13200 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:01:19.463828 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:01:19.463892 13200 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:01:19.464771 13204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:01:19.480358 13204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:01:19.482105 13204 catalog_manager.cc:1259] Loaded cluster ID: de7c52ce20084fbbb9d91682ac5e47dd
I20260806 09:01:19.482329 13204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:01:19.488091 13204 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:01:19.490054 13204 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Loaded TSK: 0
I20260806 09:01:19.529639 13204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Generated new TSK 1
I20260806 09:01:19.530081 13204 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:01:19.772856 13197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:19.773149 13197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:19.773270 13197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:19.780865 13197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:19.780998 13197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:19.781144 13197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:01:19.791424 13197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:33535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=35779
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:19.792836 13197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:19.794231 13197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:19.808723 13232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:19.808794 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:19.808815 13236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:19.810086 13197 server_base.cc:1034] running on GCE node
I20260806 09:01:19.810901 13197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:19.812137 13197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:19.813381 13197 hybrid_clock.cc:648] HybridClock initialized: now 1786006879813314 us; error 55 us; skew 500 ppm
I20260806 09:01:19.813756 13197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:19 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:19.817842 13197 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:01:19.819629 13197 webserver.cc:460] Webserver started at http://127.22.98.1:35779/ using document root <none> and password file <none>
I20260806 09:01:19.820535 13197 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:19.820690 13197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:19.827133 13197 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:01:19.830222 13252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:19.831724 13197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:01:19.831889 13197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "319226ebee8b40698e938768c4bbb083"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "908316f81f6a54f91b55f5ddeeffc01b"
server_key_iv: "e6bfe209e7d0b61815f0f2c186b52a6a"
server_key_version: "encryptionkey@0"
I20260806 09:01:19.832943 13197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:19.847059 13197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:19.851488 13197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:19.851785 13197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:19.852829 13197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:19.854247 13197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:01:19.854346 13197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:19.854450 13197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:01:19.854519 13197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:19.885331 13197 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:33535
I20260806 09:01:19.885396 13424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:33535 every 8 connection(s)
I20260806 09:01:19.886795 13197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:01:19.891891 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13197
I20260806 09:01:19.892524 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:39195
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=33165
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:01:19 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:01:19.911391 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:19.889703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:47037 (local address 127.22.98.62:34307)
0806 09:01:19.890417 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:19.890434 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:19.890748 (+ 314us) server_negotiation.cc:407] Connection header received
0806 09:01:19.892928 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:19.892947 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:19.893408 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:19.893802 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:19.895253 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:19.896963 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:19.898724 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:19.899811 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:19.900031 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:19.903823 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:19.903834 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:19.903856 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:19.903902 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:19.906966 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:19.907879 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:19.907886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:19.907893 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:19.908069 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:19.908528 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:19.908531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:19.908533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:19.909033 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:19.909186 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:19.910063 (+ 877us) server_negotiation.cc:299] Negotiation successful
0806 09:01:19.910408 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":263,"threads_started":1}
I20260806 09:01:19.911835 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:19.889788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:47037)
0806 09:01:19.890574 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:19.890624 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:19.892323 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:19.893941 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:19.893949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:19.894492 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:01:19.895029 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:19.895048 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:19.897162 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:19.897166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:19.898568 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:19.898585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:19.900085 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:19.900092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:19.900315 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:19.901044 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:01:19.901070 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:01:19.903481 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:01:19.907098 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:19.907107 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:19.907130 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:19.907724 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:19.908182 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:19.908186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:19.908190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:19.908425 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:19.909180 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:01:19.909188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:01:19.909589 (+ 401us) client_negotiation.cc:769] Sending connection context
0806 09:01:19.909888 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 09:01:19.910448 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":353,"threads_started":1}
I20260806 09:01:19.915084 13425 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:01:19.915396 13425 heartbeater.cc:464] Registering TS with master...
I20260806 09:01:19.916383 13425 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:01:19.918926 13141 ts_manager.cc:194] Registered new tserver with Master: 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535)
I20260806 09:01:19.921097 13141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:47037
W20260806 09:01:20.244508 13432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:20.244817 13432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:20.244910 13432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:20.252177 13432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:20.252389 13432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:20.252530 13432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:01:20.262879 13432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:39195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=33165
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:20.264254 13432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:20.265610 13432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:20.278422 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:20.278465 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:20.278491 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:20.279582 13432 server_base.cc:1034] running on GCE node
I20260806 09:01:20.280230 13432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:20.281155 13432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:20.282379 13432 hybrid_clock.cc:648] HybridClock initialized: now 1786006880282327 us; error 42 us; skew 500 ppm
I20260806 09:01:20.282701 13432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:20 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006880, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:20.285893 13432 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:01:20.287487 13432 webserver.cc:460] Webserver started at http://127.22.98.2:33165/ using document root <none> and password file <none>
I20260806 09:01:20.288216 13432 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:20.288353 13432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:20.293929 13432 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 09:01:20.296900 13452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.298494 13432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:01:20.298668 13432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "420486162cda414787cbce4df4dcc874"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "c05e3c35eba3cc43c69b330692238a5e"
server_key_iv: "c87ba7dfb89174e90278485fa02dbb40"
server_key_version: "encryptionkey@0"
I20260806 09:01:20.299844 13432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:20.338007 13432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:20.342670 13432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:20.342968 13432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:20.344065 13432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:20.345520 13432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:01:20.345630 13432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.345752 13432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:01:20.345826 13432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.376396 13432 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39195
I20260806 09:01:20.376457 13614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39195 every 8 connection(s)
I20260806 09:01:20.377900 13432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:01:20.379292 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13432
I20260806 09:01:20.380144 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:33687
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=41503
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:01:20 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006880, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:01:20.400707 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.381723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:60833 (local address 127.22.98.62:34307)
0806 09:01:20.381940 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.381944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.383027 (+ 1083us) server_negotiation.cc:407] Connection header received
0806 09:01:20.384951 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.384957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.385045 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.385135 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:20.386690 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.387400 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.389100 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.389584 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.389700 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.393595 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:20.393603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:20.393608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:20.393648 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:20.397045 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.398011 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.398017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.398020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.398184 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.398743 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.398749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.398752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.399120 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:20.399291 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.400096 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.400367 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 09:01:20.401938 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.381953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:60833)
0806 09:01:20.382851 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:20.382898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:20.384662 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:20.385398 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:20.385407 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:20.385940 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:01:20.386511 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.386529 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.387598 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.387602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.388956 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.388969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.389820 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.389828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.390035 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.390668 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:01:20.390696 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:01:20.393331 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:01:20.397206 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:20.397215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:20.397232 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:20.397847 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:20.398336 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:20.398341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:20.398345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:20.398586 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:20.399233 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:01:20.399241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:01:20.399687 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 09:01:20.399977 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 09:01:20.400562 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":350,"threads_started":1}
I20260806 09:01:20.403633 13617 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:01:20.403961 13617 heartbeater.cc:464] Registering TS with master...
I20260806 09:01:20.404927 13617 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:01:20.406651 13141 ts_manager.cc:194] Registered new tserver with Master: 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195)
I20260806 09:01:20.407765 13141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:60833
W20260806 09:01:20.728590 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:20.728911 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:20.729046 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:20.736527 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:20.736665 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:20.736820 13623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:01:20.747026 13623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:33687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=41503
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:20.748420 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:20.749789 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:20.763306 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:20.763248 13630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:20.763279 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:20.764055 13623 server_base.cc:1034] running on GCE node
I20260806 09:01:20.764681 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:20.765630 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:20.766862 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786006880766804 us; error 44 us; skew 500 ppm
I20260806 09:01:20.767187 13623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:20 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006880, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:20.770404 13623 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:01:20.771951 13623 webserver.cc:460] Webserver started at http://127.22.98.3:41503/ using document root <none> and password file <none>
I20260806 09:01:20.772636 13623 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:20.772778 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:20.778469 13623 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:01:20.781572 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.783044 13623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:01:20.783228 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "3bd50ea1b377fa360c4f6fd33755b1e8"
server_key_iv: "1f690e59a65daa3fbaa05b9ffbdfae5b"
server_key_version: "encryptionkey@0"
I20260806 09:01:20.784317 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:20.810683 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:20.815039 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:20.815335 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:20.816541 13623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:20.817970 13623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:01:20.818097 13623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.818213 13623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:01:20.818302 13623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:20.849756 13623 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:33687
I20260806 09:01:20.849825 13796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:33687 every 8 connection(s)
I20260806 09:01:20.851220 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:01:20.854962 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13623
Aug 06 09:01:20 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006880, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:01:20.872012 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.855942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.22.98.62:34307)
0806 09:01:20.856988 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.856992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.857008 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:01:20.857060 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.857064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.857122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.857220 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:20.858974 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.859887 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.861370 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.861855 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.861973 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.865727 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:20.865735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:20.865740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:20.865789 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:20.869179 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.869903 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.869913 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.869917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.870082 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.870428 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.870433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.870436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.870748 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:20.870955 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.871407 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.871647 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"spinlock_wait_cycles":1280,"thread_start_us":831,"threads_started":1}
I20260806 09:01:20.873868 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.853820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:51491 (local address 127.22.98.62:34307)
0806 09:01:20.854010 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.854015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.855233 (+ 1218us) server_negotiation.cc:407] Connection header received
0806 09:01:20.857586 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.857592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.857668 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.857753 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:20.859493 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.860341 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.862188 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.862690 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.862815 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.867326 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:20.867333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:20.867339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:20.867392 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:20.870380 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.871212 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.871215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.871217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.871353 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.871978 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.871983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.871986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.872357 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:20.872506 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.873300 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.873522 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:01:20.874908 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.854268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:51491)
0806 09:01:20.855053 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:20.855113 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:20.857303 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:20.858038 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:20.858047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:20.858623 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:01:20.859289 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.859310 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.860522 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.860526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.862041 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.862060 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.862982 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.862991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.863320 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.864038 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:01:20.864072 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:01:20.867071 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:01:20.870521 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:20.870529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:20.870545 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:20.871101 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:20.871469 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:20.871474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:20.871477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:20.871853 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:20.872499 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:01:20.872508 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:01:20.872932 (+ 424us) client_negotiation.cc:769] Sending connection context
0806 09:01:20.873196 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 09:01:20.873718 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":323,"threads_started":1}
I20260806 09:01:20.876241 13798 heartbeater.cc:347] Connected to a master server at 127.22.98.62:34307
I20260806 09:01:20.876541 13798 heartbeater.cc:464] Registering TS with master...
I20260806 09:01:20.877262 13798 heartbeater.cc:511] Master 127.22.98.62:34307 requested a full tablet report, sending...
I20260806 09:01:20.878708 13141 ts_manager.cc:194] Registered new tserver with Master: cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
I20260806 09:01:20.879565 13141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:51491
I20260806 09:01:20.887341 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:01:20.897537 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.888379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.22.98.62:34307)
0806 09:01:20.888571 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.888585 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.892093 (+ 3508us) server_negotiation.cc:407] Connection header received
0806 09:01:20.892326 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.892330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.892476 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.892610 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:01:20.893093 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.893874 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.895405 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.895886 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.895999 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.896523 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:01:20.896817 (+ 294us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 09:01:20.897227 (+ 410us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42014: token expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 09:01:20.897696 13429 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42014: token expired
W20260806 09:01:20.898168 13026 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.98.62:34307)
I20260806 09:01:20.910950 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.898693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.22.98.62:34307)
0806 09:01:20.898890 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.898894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.898911 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:01:20.899057 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.899060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.899142 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.899228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:20.900633 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.901437 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.902757 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.903167 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.903261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.905979 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:20.905985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:20.905988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:20.906024 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:20.908597 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.909202 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.909205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.909208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.909332 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:20.909671 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:20.909675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:20.909677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:20.910007 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:20.910198 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.910445 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.910610 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 09:01:20.914448 13141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:01:20.922045 13141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:01:20.924515 13425 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
I20260806 09:01:20.959616 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.934251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.22.98.2:39195)
0806 09:01:20.948808 (+ 14557us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.948819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.948855 (+ 36us) server_negotiation.cc:407] Connection header received
0806 09:01:20.948973 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.948983 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.949304 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.949529 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:01:20.951453 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.953082 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.956424 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.957607 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.958004 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.959123 (+ 1119us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.959191 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.959312 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14349,"thread_start_us":16549,"threads_started":1}
I20260806 09:01:20.960311 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.934043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39195 (local address 127.0.0.1:33500)
0806 09:01:20.946108 (+ 12065us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:20.946167 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:20.946377 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:20.950107 (+ 3730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:20.950115 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:20.950162 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:01:20.951043 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.951065 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.953278 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.953284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.956280 (+ 2996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.956293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.958395 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.958402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.958618 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.958707 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 09:01:20.958848 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 09:01:20.959878 (+ 1030us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11797,"mutex_wait_us":5,"thread_start_us":20052,"threads_started":1}
I20260806 09:01:20.962656 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.934470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57908)
0806 09:01:20.954411 (+ 19941us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:20.954433 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:20.954575 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:20.955296 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:20.955301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:20.955327 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:01:20.955809 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.955821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.957668 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.957673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.959990 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.960005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.961810 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.961817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.961989 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.962023 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:01:20.962113 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 09:01:20.962236 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19791,"mutex_wait_us":72,"thread_start_us":11201,"threads_started":1}
I20260806 09:01:20.962679 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.935267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.22.98.1:33535)
0806 09:01:20.953203 (+ 17936us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.953212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.954646 (+ 1434us) server_negotiation.cc:407] Connection header received
0806 09:01:20.954758 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.954767 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.955077 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.955290 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:01:20.955978 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.957502 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.960171 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.961594 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.961718 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.962019 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.962190 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.962351 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17505,"thread_start_us":17611,"threads_started":1}
I20260806 09:01:20.963141 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.934815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:33687 (local address 127.0.0.1:34906)
0806 09:01:20.954598 (+ 19783us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:20.954615 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:20.954815 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:20.955646 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:20.955651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:20.955678 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:01:20.956088 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.956101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.958050 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.958055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.960342 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:20.960356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.962419 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.962426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:20.962601 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.962633 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:01:20.962718 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 09:01:20.962841 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19670,"mutex_wait_us":40,"thread_start_us":18807,"threads_started":1}
I20260806 09:01:20.963925 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:20.935490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.22.98.3:33687)
0806 09:01:20.945420 (+ 9930us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:20.945432 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:20.954717 (+ 9285us) server_negotiation.cc:407] Connection header received
0806 09:01:20.955008 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:20.955016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:20.955205 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:20.955399 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:01:20.956321 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.957868 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.960518 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:20.961889 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:20.962005 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:20.962245 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:20.963400 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0806 09:01:20.963582 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9691,"thread_start_us":9454,"threads_started":1}
I20260806 09:01:20.964895 13520 tablet_service.cc:1467] Processing CreateTablet for tablet 764b758424bc45188e818502c7242483 (DEFAULT_TABLE table=test-table [id=bcdbe5ed8da949898150e05e7e6093c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:01:20.966506 13520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764b758424bc45188e818502c7242483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:01:20.966697 13324 tablet_service.cc:1467] Processing CreateTablet for tablet 764b758424bc45188e818502c7242483 (DEFAULT_TABLE table=test-table [id=bcdbe5ed8da949898150e05e7e6093c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:01:20.967895 13324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764b758424bc45188e818502c7242483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:01:20.967518 13704 tablet_service.cc:1467] Processing CreateTablet for tablet 764b758424bc45188e818502c7242483 (DEFAULT_TABLE table=test-table [id=bcdbe5ed8da949898150e05e7e6093c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:01:20.968935 13704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 764b758424bc45188e818502c7242483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:01:20.979804 13821 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Bootstrap starting.
I20260806 09:01:20.982390 13823 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Bootstrap starting.
I20260806 09:01:20.983700 13821 tablet_bootstrap.cc:654] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Neither blocks nor log segments found. Creating new log.
I20260806 09:01:20.984884 13824 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Bootstrap starting.
I20260806 09:01:20.985877 13821 log.cc:824] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:20.986534 13823 tablet_bootstrap.cc:654] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Neither blocks nor log segments found. Creating new log.
I20260806 09:01:20.988302 13824 tablet_bootstrap.cc:654] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Neither blocks nor log segments found. Creating new log.
I20260806 09:01:20.988229 13823 log.cc:824] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:20.989499 13824 log.cc:824] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:20.991489 13823 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: No bootstrap required, opened a new log
I20260806 09:01:20.991710 13821 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: No bootstrap required, opened a new log
I20260806 09:01:20.991784 13823 ts_tablet_manager.cc:1397] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260806 09:01:20.992024 13821 ts_tablet_manager.cc:1397] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260806 09:01:20.992595 13824 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: No bootstrap required, opened a new log
I20260806 09:01:20.992857 13824 ts_tablet_manager.cc:1397] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 09:01:20.999217 13823 raft_consensus.cc:348] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:20.999593 13823 raft_consensus.cc:374] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:01:20.999686 13823 raft_consensus.cc:729] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420486162cda414787cbce4df4dcc874, State: Initialized, Role: FOLLOWER
I20260806 09:01:21.000252 13823 consensus_queue.cc:260] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.001296 13617 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
I20260806 09:01:21.001948 13823 ts_tablet_manager.cc:1428] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260806 09:01:21.003178 13821 raft_consensus.cc:348] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.003183 13824 raft_consensus.cc:348] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.003677 13824 raft_consensus.cc:374] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:01:21.003677 13821 raft_consensus.cc:374] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:01:21.003787 13821 raft_consensus.cc:729] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319226ebee8b40698e938768c4bbb083, State: Initialized, Role: FOLLOWER
I20260806 09:01:21.003785 13824 raft_consensus.cc:729] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec819cf4a8a4b768b3cbf23f8ff83a8, State: Initialized, Role: FOLLOWER
I20260806 09:01:21.004467 13824 consensus_queue.cc:260] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.004482 13821 consensus_queue.cc:260] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.006459 13821 ts_tablet_manager.cc:1428] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 09:01:21.006614 13824 ts_tablet_manager.cc:1428] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260806 09:01:21.007349 13798 heartbeater.cc:503] Master 127.22.98.62:34307 was elected leader, sending a full tablet report...
W20260806 09:01:21.102965 13799 tablet.cc:2369] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:01:21.130085 13619 tablet.cc:2369] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:01:21.139127 13427 tablet.cc:2369] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:01:21.170149 13833 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:21.170459 13833 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.172324 13833 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
I20260806 09:01:21.178974 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.172772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:56593 (local address 127.22.98.2:39195)
0806 09:01:21.172986 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.172993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.173708 (+ 715us) server_negotiation.cc:407] Connection header received
0806 09:01:21.173898 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.173902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.173966 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.174042 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:01:21.174649 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.175369 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.177222 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.178008 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.178141 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.178254 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.178569 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.178711 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:01:21.179131 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.172957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:45843 (local address 127.22.98.3:33687)
0806 09:01:21.173170 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.173176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.173877 (+ 701us) server_negotiation.cc:407] Connection header received
0806 09:01:21.174042 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.174046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.174105 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.174180 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:01:21.174849 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.175596 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.177347 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.178165 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.178294 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.178402 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.178727 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.178875 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:01:21.179227 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.172853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:33687 (local address 127.22.98.1:45843)
0806 09:01:21.173730 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.173770 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:21.173908 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:21.174226 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:21.174230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:21.174370 (+ 140us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:01:21.174680 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:21.174690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.175735 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.175739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:21.177172 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:21.177185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.178431 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.178439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:21.178566 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.178591 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 09:01:21.178677 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 09:01:21.178812 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"spinlock_wait_cycles":26880,"thread_start_us":538,"threads_started":1}
I20260806 09:01:21.179612 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.172760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39195 (local address 127.22.98.1:56593)
0806 09:01:21.173532 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.173552 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:21.173743 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:21.174075 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:21.174079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:21.174104 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:01:21.174491 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:21.174506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.175505 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.175509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:21.177052 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:21.177069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.178247 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.178253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:21.178411 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.178438 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 09:01:21.178523 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 09:01:21.178682 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":408,"threads_started":1}
I20260806 09:01:21.180514 13736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764b758424bc45188e818502c7242483" candidate_uuid: "319226ebee8b40698e938768c4bbb083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" is_pre_election: true
I20260806 09:01:21.180675 13552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764b758424bc45188e818502c7242483" candidate_uuid: "319226ebee8b40698e938768c4bbb083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420486162cda414787cbce4df4dcc874" is_pre_election: true
I20260806 09:01:21.181233 13736 raft_consensus.cc:2454] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 319226ebee8b40698e938768c4bbb083 in term 0.
I20260806 09:01:21.181232 13552 raft_consensus.cc:2454] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 319226ebee8b40698e938768c4bbb083 in term 0.
I20260806 09:01:21.182020 13254 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 319226ebee8b40698e938768c4bbb083, 420486162cda414787cbce4df4dcc874; no voters:
I20260806 09:01:21.182451 13833 raft_consensus.cc:2781] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:01:21.182587 13833 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:01:21.182686 13833 raft_consensus.cc:3037] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:01:21.185477 13833 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.186134 13833 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 1 election: Requested vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
I20260806 09:01:21.186697 13552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764b758424bc45188e818502c7242483" candidate_uuid: "319226ebee8b40698e938768c4bbb083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420486162cda414787cbce4df4dcc874"
I20260806 09:01:21.186753 13736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "764b758424bc45188e818502c7242483" candidate_uuid: "319226ebee8b40698e938768c4bbb083" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
I20260806 09:01:21.186939 13552 raft_consensus.cc:3037] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:01:21.187000 13736 raft_consensus.cc:3037] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:01:21.189384 13552 raft_consensus.cc:2454] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 319226ebee8b40698e938768c4bbb083 in term 1.
I20260806 09:01:21.189452 13736 raft_consensus.cc:2454] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 319226ebee8b40698e938768c4bbb083 in term 1.
I20260806 09:01:21.189827 13254 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 319226ebee8b40698e938768c4bbb083, 420486162cda414787cbce4df4dcc874; no voters:
I20260806 09:01:21.190166 13833 raft_consensus.cc:2781] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:01:21.190619 13833 raft_consensus.cc:686] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 LEADER]: Becoming Leader. State: Replica: 319226ebee8b40698e938768c4bbb083, State: Running, Role: LEADER
I20260806 09:01:21.191155 13833 consensus_queue.cc:237] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:21.195833 13141 catalog_manager.cc:5495] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 reported cstate change: term changed from 0 to 1, leader changed from <none> to 319226ebee8b40698e938768c4bbb083 (127.22.98.1). New cstate: current_term: 1 leader_uuid: "319226ebee8b40698e938768c4bbb083" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } health_report { overall_health: UNKNOWN } } }
I20260806 09:01:21.255506 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.249851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.22.98.1:33535)
0806 09:01:21.250089 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.250095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.250118 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:01:21.250258 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.250262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.250361 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.250504 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:01:21.250981 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.251793 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.253314 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.253728 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.253842 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.254358 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:01:21.254897 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:01:21.255036 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.255157 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.255246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 09:01:21.264606 13736 raft_consensus.cc:1261] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Refusing update from remote peer 319226ebee8b40698e938768c4bbb083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:01:21.264606 13552 raft_consensus.cc:1261] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Refusing update from remote peer 319226ebee8b40698e938768c4bbb083: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:01:21.265658 13842 consensus_queue.cc:1035] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:01:21.266068 13833 consensus_queue.cc:1035] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [LEADER]: Connected to new peer: Peer: permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:01:21.277429 13845 mvcc.cc:205] Tried to move back new op lower bound from 7315484185649201152 to 7315484185362837504. Current Snapshot: MvccSnapshot[applied={T|T < 7315484185649201152}]
I20260806 09:01:21.291416 13141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:01:21.291718 13141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:01:21.295269 13141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Sending DeleteTablet for 3 replicas of tablet 764b758424bc45188e818502c7242483
I20260806 09:01:21.296599 13520 tablet_service.cc:1514] Processing DeleteTablet for tablet 764b758424bc45188e818502c7242483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:01:21 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:33500
I20260806 09:01:21.296599 13704 tablet_service.cc:1514] Processing DeleteTablet for tablet 764b758424bc45188e818502c7242483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:01:21 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 09:01:21.296600 13324 tablet_service.cc:1514] Processing DeleteTablet for tablet 764b758424bc45188e818502c7242483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:01:21 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260806 09:01:21.298173 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 13197
I20260806 09:01:21.314474 13862 tablet_replica.cc:330] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: stopping tablet replica
I20260806 09:01:21.315157 13863 tablet_replica.cc:330] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: stopping tablet replica
I20260806 09:01:21.315899 13863 raft_consensus.cc:2229] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:01:21.316497 13862 raft_consensus.cc:2229] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:01:21.317179 13863 raft_consensus.cc:2258] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:01:21.317752 13862 raft_consensus.cc:2258] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:01:21.320446 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.299660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57938 (local address 127.22.98.1:33535)
0806 09:01:21.299866 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.299873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.299896 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:01:21.299974 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.299979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.300081 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.300211 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:21.302535 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.303937 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.306258 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.307015 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.307118 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.311169 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:21.311192 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:21.311219 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:21.311279 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:21.316112 (+ 4833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.317145 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.317155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.317159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.317359 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.317850 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.317856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.317859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.318366 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:21.318651 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.319738 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.320043 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:01:21.321748 13420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:21.322737 13863 ts_tablet_manager.cc:1905] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:01:21.323464 13862 ts_tablet_manager.cc:1905] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:01:21.331081 13863 ts_tablet_manager.cc:1918] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:01:21.331099 13862 ts_tablet_manager.cc:1918] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:01:21.331327 13863 log.cc:1196] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/764b758424bc45188e818502c7242483
I20260806 09:01:21.331465 13862 log.cc:1196] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/764b758424bc45188e818502c7242483
I20260806 09:01:21.331984 13863 ts_tablet_manager.cc:1939] T 764b758424bc45188e818502c7242483 P 420486162cda414787cbce4df4dcc874: Deleting consensus metadata
I20260806 09:01:21.332202 13862 ts_tablet_manager.cc:1939] T 764b758424bc45188e818502c7242483 P cec819cf4a8a4b768b3cbf23f8ff83a8: Deleting consensus metadata
I20260806 09:01:21.334164 13118 catalog_manager.cc:4841] TS cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): tablet 764b758424bc45188e818502c7242483 (table test-table [id=bcdbe5ed8da949898150e05e7e6093c2]) successfully deleted
I20260806 09:01:21.334649 13116 catalog_manager.cc:4841] TS 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): tablet 764b758424bc45188e818502c7242483 (table test-table [id=bcdbe5ed8da949898150e05e7e6093c2]) successfully deleted
I20260806 09:01:21.432530 13860 tablet_replica.cc:330] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: stopping tablet replica
I20260806 09:01:21.433005 13860 raft_consensus.cc:2229] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:01:21.433512 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 13197
W20260806 09:01:21.456786 13120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:33535 (error 108)
I20260806 09:01:21.456925 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 13432
I20260806 09:01:21.458783 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.458267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57940)
0806 09:01:21.458457 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.458622 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 09:01:21.459017 13120 catalog_manager.cc:4584] TS 319226ebee8b40698e938768c4bbb083 (127.22.98.1:33535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 764b758424bc45188e818502c7242483: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
I20260806 09:01:21.471118 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.457812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.22.98.2:39195)
0806 09:01:21.458000 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.458007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.458100 (+ 93us) server_negotiation.cc:407] Connection header received
0806 09:01:21.458298 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.458304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.458390 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.458495 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:21.460116 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.460915 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.462451 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.462862 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.462956 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.465654 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:21.465670 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:21.465689 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:21.465725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:21.468589 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.469271 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.469278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.469280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.469404 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.469723 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.469727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.469729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.470125 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:21.470364 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.470596 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.470773 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:01:21.471819 13610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:21.512569 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.512093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57950)
0806 09:01:21.512319 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.512420 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 09:01:21.513664 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.513297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57954)
0806 09:01:21.513453 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.513533 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 09:01:21.560546 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 13432
I20260806 09:01:21.574481 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.573848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57968)
0806 09:01:21.574103 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.574228 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 09:01:21.575686 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.575220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57972)
0806 09:01:21.575394 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.575496 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 09:01:21.581897 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 13623
I20260806 09:01:21.594842 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.582631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.22.98.3:33687)
0806 09:01:21.582801 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:21.582805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:21.582841 (+ 36us) server_negotiation.cc:407] Connection header received
0806 09:01:21.583000 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:21.583004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:21.583070 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:21.583153 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:21.584527 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.585230 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.586538 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:21.586929 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:21.587018 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:21.589615 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:21.589630 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:21.589649 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:21.589687 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:21.592255 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.592952 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.592960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.592962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.593101 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:21.593503 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:21.593506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:21.593508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:21.593860 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:21.594066 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:21.594379 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 09:01:21.594577 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:01:21.595490 13792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:21.656494 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.655900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57976)
0806 09:01:21.656145 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.656262 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 09:01:21.657728 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:21.657259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:33535 (local address 127.0.0.1:57982)
0806 09:01:21.657497 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:21.657589 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 09:01:21.684182 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 13623
I20260806 09:01:21.707871 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 13091
I20260806 09:01:21.709215 13189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:21.824743 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 13091
I20260806 09:01:41.850440 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--webserver_interface=127.22.98.62
--webserver_port=39311
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:01:42.209329 13870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:42.209712 13870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:42.209877 13870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:42.217293 13870 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:01:42.217425 13870 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:01:42.217483 13870 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:01:42.217555 13870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:01:42.217640 13870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:42.217741 13870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:42.217805 13870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:01:42.217881 13870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:01:42.228310 13870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=39311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:42.229758 13870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:42.231124 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:42.244211 13878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:42.244294 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:42.244238 13880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:42.244554 13870 server_base.cc:1034] running on GCE node
I20260806 09:01:42.245820 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:42.247416 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:42.248733 13870 hybrid_clock.cc:648] HybridClock initialized: now 1786006902248638 us; error 60 us; skew 500 ppm
I20260806 09:01:42.249115 13870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:42 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006902, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:42.252526 13870 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:01:42.254099 13870 webserver.cc:460] Webserver started at http://127.22.98.62:39311/ using document root <none> and password file <none>
I20260806 09:01:42.254829 13870 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:42.254923 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:42.261195 13870 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 09:01:42.264668 13892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:42.266297 13870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:01:42.266508 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "e752207e9bae4fbda2fb702e1d5a909d"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "e8cb8876af9512f36d0d5106c2043e1d"
server_key_iv: "64859c0f0dbb112f541b5f8402fa3bdf"
server_key_version: "encryptionkey@0"
I20260806 09:01:42.267678 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:42.299523 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:42.304339 13870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:42.304647 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:42.324095 13870 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34307
I20260806 09:01:42.324055 13967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34307 every 8 connection(s)
I20260806 09:01:42.325795 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:01:42.333356 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13870
I20260806 09:01:42.333437 13968 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:01:42.334297 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:33535
--local_ip_for_outbound_sockets=127.22.98.1
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=35779
--webserver_interface=127.22.98.1
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:01:42.337191 13968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap starting.
I20260806 09:01:42.359341 13968 log.cc:824] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:42.377048 13968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:01:42.377700 13968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Bootstrap complete.
I20260806 09:01:42.387225 13968 raft_consensus.cc:348] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:42.388314 13968 raft_consensus.cc:729] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Initialized, Role: FOLLOWER
I20260806 09:01:42.388996 13968 consensus_queue.cc:260] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:42.389242 13968 raft_consensus.cc:388] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:01:42.389358 13968 raft_consensus.cc:482] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:01:42.389526 13968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:01:42.393527 13968 raft_consensus.cc:504] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:42.394032 13968 leader_election.cc:302] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e752207e9bae4fbda2fb702e1d5a909d; no voters:
I20260806 09:01:42.394706 13968 leader_election.cc:288] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 09:01:42.394846 13972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 09:01:42.396132 13972 raft_consensus.cc:686] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [term 3 LEADER]: Becoming Leader. State: Replica: e752207e9bae4fbda2fb702e1d5a909d, State: Running, Role: LEADER
I20260806 09:01:42.396775 13972 consensus_queue.cc:237] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } }
I20260806 09:01:42.397243 13968 sys_catalog.cc:564] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:01:42.399581 13974 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:01:42.399997 13974 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:01:42.403829 13977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e752207e9bae4fbda2fb702e1d5a909d. Latest consensus state: current_term: 3 leader_uuid: "e752207e9bae4fbda2fb702e1d5a909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e752207e9bae4fbda2fb702e1d5a909d" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34307 } } }
I20260806 09:01:42.404179 13977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d [sys.catalog]: This master's current role is: LEADER
I20260806 09:01:42.405833 13981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:01:42.414652 13981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:01:42.416324 13981 catalog_manager.cc:1259] Loaded cluster ID: de7c52ce20084fbbb9d91682ac5e47dd
I20260806 09:01:42.416445 13981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:01:42.421155 13981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:01:42.423069 13981 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Loaded TSK: 0
I20260806 09:01:42.423254 13981 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e752207e9bae4fbda2fb702e1d5a909d: Loaded TSK: 1
F20260806 09:01:42.424535 13981 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f04d9e299ad google::LogMessage::Fail() at ??:0
@ 0x7f04d9e2c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f04d9e294de google::LogMessage::Flush() at ??:0
@ 0x7f04d9e2aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f04ddb8d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f04ddb7c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f04ef8581c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f04ef857783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f04ef8e32a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f04ef8e30c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f04ef92d5e9 std::function<>::operator()() at ??:0
@ 0x7f04ef85bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f04ef859c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f04ef8e1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f04ef8e1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f04dca75345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f04dca7891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f04dca78799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f04dca53eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f04dfd2d6db start_thread at ??:0
@ 0x7f04d8cc361f clone at ??:0
W20260806 09:01:42.771926 13970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:42.772240 13970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:42.772373 13970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:42.780046 13970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:42.780186 13970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:42.780340 13970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:01:42.790625 13970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:33535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=35779
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:42.792018 13970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:42.793394 13970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:42.806739 14045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:42.806998 14044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:42.807478 14047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:42.808063 13970 server_base.cc:1034] running on GCE node
I20260806 09:01:42.808836 13970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:42.809863 13970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:42.811088 13970 hybrid_clock.cc:648] HybridClock initialized: now 1786006902811020 us; error 57 us; skew 500 ppm
I20260806 09:01:42.811430 13970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:42 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006902, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:42.814576 13970 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:01:42.816188 13970 webserver.cc:460] Webserver started at http://127.22.98.1:35779/ using document root <none> and password file <none>
I20260806 09:01:42.816880 13970 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:42.816995 13970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:42.822980 13970 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 09:01:42.826489 14056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:42.828147 13970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:01:42.828328 13970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "319226ebee8b40698e938768c4bbb083"
format_stamp: "Formatted at 2026-08-06 09:00:56 on dist-test-slave-jpj1"
server_key: "908316f81f6a54f91b55f5ddeeffc01b"
server_key_iv: "e6bfe209e7d0b61815f0f2c186b52a6a"
server_key_version: "encryptionkey@0"
I20260806 09:01:42.829413 13970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:42.850265 13970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:42.854734 13970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:42.855013 13970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:42.856163 13970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:42.858115 14066 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 09:01:42.864197 13970 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 09:01:42.864324 13970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 09:01:42.864457 13970 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 09:01:42.867779 13970 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 09:01:42.867866 13970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260806 09:01:42.868204 14066 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Bootstrap starting.
I20260806 09:01:42.896124 14066 log.cc:824] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Log is configured to *not* fsync() on all Append() calls
I20260806 09:01:42.904959 13970 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:33535
I20260806 09:01:42.904974 14224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:33535 every 8 connection(s)
I20260806 09:01:42.906606 13970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:01:42.911396 14066 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:01:42.911947 14066 tablet_bootstrap.cc:492] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Bootstrap complete.
I20260806 09:01:42.912081 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:42.909708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:47133)
0806 09:01:42.910623 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:42.910979 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":438,"threads_started":1}
W20260806 09:01:42.912721 14225 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:34307 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:34307: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
I20260806 09:01:42.913547 14066 ts_tablet_manager.cc:1397] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Time spent bootstrapping tablet: real 0.046s user 0.026s sys 0.016s
I20260806 09:01:42.913627 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:42.913246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:52269)
0806 09:01:42.913418 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:42.913492 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 09:01:42.914559 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:42.914182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:37097)
0806 09:01:42.914348 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:42.914429 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 09:01:42.914819 14225 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:01:42.915380 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 13970
I20260806 09:01:42.916011 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:39195
--local_ip_for_outbound_sockets=127.22.98.2
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=33165
--webserver_interface=127.22.98.2
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:01:42.923599 14066 raft_consensus.cc:348] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:42.924803 14066 raft_consensus.cc:729] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319226ebee8b40698e938768c4bbb083, State: Initialized, Role: FOLLOWER
I20260806 09:01:42.925541 14066 consensus_queue.cc:260] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:42.928054 14066 ts_tablet_manager.cc:1428] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260806 09:01:42.928256 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:42.927805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:52453)
0806 09:01:42.927992 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:42.928091 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 09:01:42.929188 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:42.928861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:51891)
0806 09:01:42.929009 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:42.929075 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 09:01:43.259343 14234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:43.259696 14234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:43.259846 14234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:43.267036 14234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:43.267158 14234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:43.267311 14234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:01:43.276829 14234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:39195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=33165
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:43.278070 14234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:43.279407 14234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:43.292382 14242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:43.292411 14243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:43.292433 14246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:43.293525 14234 server_base.cc:1034] running on GCE node
I20260806 09:01:43.294152 14234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:43.295133 14234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:43.296344 14234 hybrid_clock.cc:648] HybridClock initialized: now 1786006903296277 us; error 46 us; skew 500 ppm
I20260806 09:01:43.296651 14234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:43 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006903, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:43.300019 14234 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:01:43.301601 14234 webserver.cc:460] Webserver started at http://127.22.98.2:33165/ using document root <none> and password file <none>
I20260806 09:01:43.302318 14234 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:43.302446 14234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:43.308079 14234 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:01:43.310842 14261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.312305 14234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:01:43.312472 14234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "420486162cda414787cbce4df4dcc874"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "c05e3c35eba3cc43c69b330692238a5e"
server_key_iv: "c87ba7dfb89174e90278485fa02dbb40"
server_key_version: "encryptionkey@0"
I20260806 09:01:43.313519 14234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:43.344269 14234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:43.348954 14234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:43.349241 14234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:43.350349 14234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:43.351933 14234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:01:43.352038 14234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.352150 14234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:01:43.352228 14234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.384827 14234 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39195
I20260806 09:01:43.384914 14427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39195 every 8 connection(s)
I20260806 09:01:43.386310 14234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:01:43.387106 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 14234
I20260806 09:01:43.387897 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:33687
--local_ip_for_outbound_sockets=127.22.98.3
--tserver_master_addrs=127.22.98.62:34307
--webserver_port=41503
--webserver_interface=127.22.98.3
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:01:43.391945 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.389918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:34541)
0806 09:01:43.390730 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.390964 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":329,"threads_started":1}
W20260806 09:01:43.392683 14430 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:34307 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:34307: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
I20260806 09:01:43.393546 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.393229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:39805)
0806 09:01:43.393369 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.393439 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 09:01:43.394433 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.394076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:51213)
0806 09:01:43.394246 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.394324 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 09:01:43.394666 14430 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:01:43.734848 14436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:01:43.735137 14436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:01:43.735231 14436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:01:43.742580 14436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:01:43.742702 14436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:01:43.742825 14436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:01:43.752768 14436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:33687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=41503
--tserver_master_addrs=127.22.98.62:34307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:01:43.754089 14436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:01:43.755406 14436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:01:43.769563 14449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:43.769541 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:01:43.770236 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:01:43.770242 14436 server_base.cc:1034] running on GCE node
I20260806 09:01:43.770960 14436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:01:43.772001 14436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:01:43.773226 14436 hybrid_clock.cc:648] HybridClock initialized: now 1786006903773169 us; error 47 us; skew 500 ppm
I20260806 09:01:43.773557 14436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:01:43 dist-test-slave-jpj1 krb5kdc[12280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006903, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:01:43.776805 14436 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:01:43.778282 14436 webserver.cc:460] Webserver started at http://127.22.98.3:41503/ using document root <none> and password file <none>
I20260806 09:01:43.778998 14436 fs_manager.cc:359] Metadata directory not provided
I20260806 09:01:43.779124 14436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:01:43.784687 14436 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 09:01:43.787708 14458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.789306 14436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:01:43.789498 14436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8"
format_stamp: "Formatted at 2026-08-06 09:00:57 on dist-test-slave-jpj1"
server_key: "3bd50ea1b377fa360c4f6fd33755b1e8"
server_key_iv: "1f690e59a65daa3fbaa05b9ffbdfae5b"
server_key_version: "encryptionkey@0"
I20260806 09:01:43.790534 14436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:01:43.803002 14436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:01:43.807572 14436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:01:43.807963 14436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:01:43.809019 14436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:01:43.810527 14436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:01:43.810616 14436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.810712 14436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:01:43.810782 14436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:01:43.843240 14436 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:33687
I20260806 09:01:43.843309 14616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:33687 every 8 connection(s)
I20260806 09:01:43.844729 14436 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:01:43.849707 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.847759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:52807)
0806 09:01:43.848520 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.848779 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":291,"threads_started":1}
W20260806 09:01:43.850477 14619 heartbeater.cc:644] Failed to heartbeat to 127.22.98.62:34307 (0 consecutive failures): Network error: Failed to ping master at 127.22.98.62:34307: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
I20260806 09:01:43.850826 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 14436
I20260806 09:01:43.851378 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.850967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:32905)
0806 09:01:43.851143 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.851245 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 09:01:43.852368 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.851995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:52137)
0806 09:01:43.852137 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.852230 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 09:01:43.852670 14619 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:01:43.931205 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:43.930251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:38873)
0806 09:01:43.930830 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:43.930923 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":284,"threads_started":1}
I20260806 09:01:44.396023 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.395176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:33077)
0806 09:01:44.395688 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:44.395780 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":248,"threads_started":1}
I20260806 09:01:44.565322 14626 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:44.565609 14626 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:44.567271 14626 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
Aug 06 09:01:44 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006902, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
Aug 06 09:01:44 dist-test-slave-jpj1 krb5kdc[12280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006902, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:01:44.604206 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.567947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:47719 (local address 127.22.98.3:33687)
0806 09:01:44.580586 (+ 12639us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:44.580602 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:44.581086 (+ 484us) server_negotiation.cc:407] Connection header received
0806 09:01:44.582632 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:44.582642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:44.583072 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:44.583383 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:44.585592 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.586818 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.588641 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.589271 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.589441 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:44.597131 (+ 7690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:44.597142 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:44.597157 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:44.597197 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:44.600287 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601188 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:44.601195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:44.601202 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:44.601368 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601813 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:44.601817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:44.601819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:44.602361 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:44.602534 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:44.603383 (+ 849us) server_negotiation.cc:299] Negotiation successful
0806 09:01:44.603792 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12450,"thread_start_us":12336,"threads_started":1}
I20260806 09:01:44.604211 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.568067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:33129 (local address 127.22.98.2:39195)
0806 09:01:44.579802 (+ 11735us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:44.579823 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:44.580709 (+ 886us) server_negotiation.cc:407] Connection header received
0806 09:01:44.582751 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:44.582760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:44.583143 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:44.583439 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:44.585785 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.586968 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.588647 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.589256 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.589413 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:44.597090 (+ 7677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:44.597104 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:44.597125 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:44.597168 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:44.600391 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601201 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:44.601207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:44.601213 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:44.601368 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601820 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:44.601823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:44.601826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:44.602364 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:44.602537 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:44.603410 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 09:01:44.603833 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11518,"thread_start_us":11285,"threads_started":1}
I20260806 09:01:44.604358 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.567772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:33687 (local address 127.22.98.1:47719)
0806 09:01:44.580956 (+ 13184us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:44.580970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:44.581095 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:44.583620 (+ 2525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:44.583632 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:44.584548 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:01:44.585389 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:44.585421 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.587002 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.587007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:44.588501 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:44.588520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.589498 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.589505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:44.589755 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:44.593679 (+ 3924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:01:44.593715 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:01:44.596769 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:01:44.600450 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:44.600460 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:44.600477 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:44.601041 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:44.601478 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:44.601487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:44.601699 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:44.602506 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:01:44.602514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:01:44.602878 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 09:01:44.603149 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 09:01:44.603865 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13069,"mutex_wait_us":401,"spinlock_wait_cycles":76160,"thread_start_us":12833,"threads_started":1}
I20260806 09:01:44.605181 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.567582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39195 (local address 127.22.98.1:33129)
0806 09:01:44.580126 (+ 12544us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:44.580184 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 09:01:44.580929 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:01:44.583620 (+ 2691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:01:44.583631 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:01:44.584549 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:01:44.585389 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:44.585420 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.587137 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.587141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:44.588502 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:01:44.588521 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:01:44.589473 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:01:44.589481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:01:44.589759 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:44.593695 (+ 3936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:01:44.593716 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:01:44.596633 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:01:44.600538 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:44.600543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:44.600547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:44.601049 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:44.601482 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:01:44.601486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:01:44.601488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:01:44.601706 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:01:44.602505 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:01:44.602514 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:01:44.602873 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 09:01:44.603121 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 09:01:44.603726 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12383,"mutex_wait_us":167,"spinlock_wait_cycles":8704,"thread_start_us":12222,"threads_started":1}
W20260806 09:01:44.607767 14062 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): Not found: Tablet not found: 764b758424bc45188e818502c7242483
W20260806 09:01:44.608073 14061 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): Not found: Tablet not found: 764b758424bc45188e818502c7242483
I20260806 09:01:44.608263 14061 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 319226ebee8b40698e938768c4bbb083; no voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8
I20260806 09:01:44.608712 14626 raft_consensus.cc:2726] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:01:44.854111 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.853177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:40073)
0806 09:01:44.853710 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:44.853849 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":225,"threads_started":1}
I20260806 09:01:44.932557 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:44.932117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:38149)
0806 09:01:44.932300 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:44.932403 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 09:01:45.397608 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:45.396898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:59681)
0806 09:01:45.397357 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:45.397442 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260806 09:01:45.855859 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:45.855070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:57663)
0806 09:01:45.855597 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:45.855696 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260806 09:01:45.934425 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:45.933532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:34449)
0806 09:01:45.934143 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:45.934234 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":304,"threads_started":1}
I20260806 09:01:46.322309 14643 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:46.322535 14643 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:46.323205 14643 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
W20260806 09:01:46.324370 14061 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): Not found: Tablet not found: 764b758424bc45188e818502c7242483
W20260806 09:01:46.324545 14062 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): Not found: Tablet not found: 764b758424bc45188e818502c7242483
I20260806 09:01:46.324705 14062 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 319226ebee8b40698e938768c4bbb083; no voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8
I20260806 09:01:46.324947 14643 raft_consensus.cc:2726] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:01:46.399328 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:46.398483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:58313)
0806 09:01:46.398987 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:46.399104 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260806 09:01:46.857586 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:46.856830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:54227)
0806 09:01:46.857314 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:46.857407 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260806 09:01:46.936283 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:46.935457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:44287)
0806 09:01:46.935992 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:46.936085 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":292,"threads_started":1}
I20260806 09:01:47.401058 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:47.400313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:55889)
0806 09:01:47.400753 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:47.400849 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":217,"threads_started":1}
I20260806 09:01:47.859444 14650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:47.858636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:45999)
0806 09:01:47.859135 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:47.859229 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260806 09:01:47.938200 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:47.937380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:48679)
0806 09:01:47.937905 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:47.938000 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":294,"threads_started":1}
I20260806 09:01:48.403115 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:48.402185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:54073)
0806 09:01:48.402770 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:48.402881 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":281,"threads_started":1}
I20260806 09:01:48.861189 14656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:48.860411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:45801)
0806 09:01:48.860914 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:48.861000 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":259,"threads_started":1}
I20260806 09:01:48.939810 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:48.939031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:49801)
0806 09:01:48.939498 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:48.939632 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":250,"threads_started":1}
I20260806 09:01:49.405170 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:49.404188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:50263)
0806 09:01:49.404789 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:49.404918 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":299,"threads_started":1}
I20260806 09:01:49.602697 14660 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:49.602903 14660 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:49.603569 14660 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
W20260806 09:01:49.604622 14061 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): Not found: Tablet not found: 764b758424bc45188e818502c7242483
W20260806 09:01:49.604801 14062 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): Not found: Tablet not found: 764b758424bc45188e818502c7242483
I20260806 09:01:49.604914 14062 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 319226ebee8b40698e938768c4bbb083; no voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8
I20260806 09:01:49.605129 14660 raft_consensus.cc:2726] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:01:49.862969 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:49.862181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:45289)
0806 09:01:49.862689 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:49.862789 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":271,"threads_started":1}
I20260806 09:01:49.941577 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:49.940778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:48749)
0806 09:01:49.941266 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:49.941361 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":229,"threads_started":1}
I20260806 09:01:50.406996 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:50.406169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:50693)
0806 09:01:50.406704 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:50.406799 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":250,"threads_started":1}
I20260806 09:01:50.864804 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:50.863946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:39303)
0806 09:01:50.864508 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:50.864594 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":257,"threads_started":1}
I20260806 09:01:50.943327 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:50.942555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:37125)
0806 09:01:50.943036 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:50.943132 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":246,"threads_started":1}
I20260806 09:01:51.408903 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:51.408110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:52277)
0806 09:01:51.408563 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:51.408662 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":225,"threads_started":1}
I20260806 09:01:51.866528 14672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:51.865765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:51765)
0806 09:01:51.866240 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:51.866332 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260806 09:01:51.945087 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:51.944281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:49219)
0806 09:01:51.944796 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:51.944886 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":258,"threads_started":1}
I20260806 09:01:52.410687 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:52.409891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:42863)
0806 09:01:52.410431 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:52.410525 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":238,"threads_started":1}
I20260806 09:01:52.738855 14675 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:52.739038 14675 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:52.739723 14675 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
W20260806 09:01:52.740617 14061 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): Not found: Tablet not found: 764b758424bc45188e818502c7242483
W20260806 09:01:52.740806 14062 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): Not found: Tablet not found: 764b758424bc45188e818502c7242483
I20260806 09:01:52.740954 14062 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 319226ebee8b40698e938768c4bbb083; no voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8
I20260806 09:01:52.741204 14675 raft_consensus.cc:2726] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:01:52.868492 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:52.867641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:44045)
0806 09:01:52.868255 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:52.868334 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":254,"threads_started":1}
I20260806 09:01:52.947346 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:52.946362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:38231)
0806 09:01:52.947042 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:52.947133 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":380,"threads_started":1}
I20260806 09:01:53.412415 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:53.411662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:51361)
0806 09:01:53.412140 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:53.412228 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260806 09:01:53.870232 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:53.869438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:44235)
0806 09:01:53.869954 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:53.870042 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":265,"threads_started":1}
I20260806 09:01:53.949112 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:53.948317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:49527)
0806 09:01:53.948828 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:53.948912 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":289,"threads_started":1}
I20260806 09:01:54.414230 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:54.413429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:45553)
0806 09:01:54.413953 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:54.414054 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
I20260806 09:01:54.872030 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:54.871161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:46567)
0806 09:01:54.871743 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:54.871844 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":262,"threads_started":1}
I20260806 09:01:54.951001 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:54.950156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:46301)
0806 09:01:54.950719 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:54.950829 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":270,"threads_started":1}
I20260806 09:01:55.416152 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:55.415280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:43297)
0806 09:01:55.415839 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:55.415937 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":288,"threads_started":1}
I20260806 09:01:55.873883 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:55.873061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:38399)
0806 09:01:55.873597 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:55.873690 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":275,"threads_started":1}
I20260806 09:01:55.953020 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:55.952158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:51259)
0806 09:01:55.952698 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:55.952795 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":291,"threads_started":1}
I20260806 09:01:56.204851 14693 raft_consensus.cc:482] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:01:56.205075 14693 raft_consensus.cc:504] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420486162cda414787cbce4df4dcc874" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39195 } } peers { permanent_uuid: "319226ebee8b40698e938768c4bbb083" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 33535 } } peers { permanent_uuid: "cec819cf4a8a4b768b3cbf23f8ff83a8" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 33687 } }
I20260806 09:01:56.205830 14693 leader_election.cc:288] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195), cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687)
W20260806 09:01:56.206808 14062 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cec819cf4a8a4b768b3cbf23f8ff83a8 (127.22.98.3:33687): Not found: Tablet not found: 764b758424bc45188e818502c7242483
W20260806 09:01:56.207017 14061 leader_election.cc:341] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 420486162cda414787cbce4df4dcc874 (127.22.98.2:39195): Not found: Tablet not found: 764b758424bc45188e818502c7242483
I20260806 09:01:56.207190 14061 leader_election.cc:302] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 319226ebee8b40698e938768c4bbb083; no voters: 420486162cda414787cbce4df4dcc874, cec819cf4a8a4b768b3cbf23f8ff83a8
I20260806 09:01:56.207497 14693 raft_consensus.cc:2726] T 764b758424bc45188e818502c7242483 P 319226ebee8b40698e938768c4bbb083 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:01:56.417934 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:56.417109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:39663)
0806 09:01:56.417643 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:56.417756 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":284,"threads_started":1}
I20260806 09:01:56.875878 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:56.875013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:38487)
0806 09:01:56.875506 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:56.875699 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":238,"threads_started":1}
I20260806 09:01:56.955286 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:56.954200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:59697)
0806 09:01:56.954838 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:56.954962 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":320,"threads_started":1}
I20260806 09:01:57.419896 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:57.418885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:36347)
0806 09:01:57.419435 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:57.419605 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":256,"threads_started":1}
I20260806 09:01:57.877552 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:57.876760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:53295)
0806 09:01:57.877236 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:57.877336 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":233,"threads_started":1}
I20260806 09:01:57.957223 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:57.956516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:42133)
0806 09:01:57.956962 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:57.957053 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":212,"threads_started":1}
I20260806 09:01:58.421859 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:58.421123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.2:36741)
0806 09:01:58.421592 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:58.421676 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260806 09:01:58.858003 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 13970
I20260806 09:01:58.877882 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:58.859145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.22.98.1:33535)
0806 09:01:58.859698 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:58.859706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:58.859733 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:01:58.859807 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:58.859814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:58.860043 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:58.860200 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:58.861733 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:58.862712 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:58.864049 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:58.864661 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:58.864796 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:58.872068 (+ 7272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:58.872086 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:58.872105 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:58.872147 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:58.874871 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:58.875639 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:58.875647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:58.875649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:58.875825 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:58.876227 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:58.876232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:58.876234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:58.876619 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:58.876929 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:58.877275 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 09:01:58.877548 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260806 09:01:58.879379 14219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:58.879456 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:58.878720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.3:44569)
0806 09:01:58.879187 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:58.879309 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":247,"threads_started":1}
I20260806 09:01:58.968856 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:58.967791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:34307 (local address 127.22.98.1:45003)
0806 09:01:58.968362 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 09:01:58.968500 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.62:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":263,"threads_started":1}
I20260806 09:01:58.969259 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 13970
I20260806 09:01:58.993993 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 14234
I20260806 09:01:59.008093 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:58.994787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.22.98.2:39195)
0806 09:01:58.995338 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:58.995342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:58.995361 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:01:58.995432 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:58.995437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:58.995519 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:58.995658 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:58.996917 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:58.997731 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:58.999310 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:58.999895 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:58.999988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:59.002740 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:59.002746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:59.002749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:59.002784 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:59.005429 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:59.006148 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:59.006152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:59.006154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:59.006297 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:59.006656 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:59.006660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:59.006663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:59.007007 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:59.007220 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:59.007467 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 09:01:59.007702 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":269,"threads_started":1}
I20260806 09:01:59.008824 14422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:59.094424 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 14234
I20260806 09:01:59.116211 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 14436
I20260806 09:01:59.133297 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:01:59.116989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.22.98.3:33687)
0806 09:01:59.117549 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 09:01:59.117554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:01:59.117572 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:01:59.117631 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:01:59.117635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:01:59.117705 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:01:59.117805 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:01:59.119171 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:59.120042 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:59.121223 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:01:59.121662 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:01:59.121772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:01:59.127728 (+ 5956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:01:59.127735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:01:59.127739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:01:59.127784 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:01:59.130487 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:59.131212 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:59.131217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:59.131219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:59.131373 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:01:59.131761 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:01:59.131765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:01:59.131768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:01:59.132087 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:01:59.132331 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 09:01:59.132733 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 09:01:59.132926 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":280,"threads_started":1}
I20260806 09:01:59.134119 14610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:01:59.219264 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 14436
2026-08-06T09:01:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64942 ms)
[----------] 30 tests from SecurityITest (308802 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:01:59 dist-test-slave-jpj1 krb5kdc[14716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:01:59 dist-test-slave-jpj1 krb5kdc[14716](info): set up 2 sockets
Aug 06 09:01:59 dist-test-slave-jpj1 krb5kdc[14716](info): commencing operation
krb5kdc: starting...
W20260806 09:02:00.441175 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:00 dist-test-slave-jpj1 krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:00.764122 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:46773
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:46773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:01.130813 14735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:01.131145 14735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:01.131271 14735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:01.138899 14735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:01.139034 14735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:01.139087 14735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:01.139173 14735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:01.139247 14735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:01.150362 14735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:46773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:46773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:01.151834 14735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:01.153179 14735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:01.165621 14744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:01.165591 14748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:01.165674 14746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:01.166015 14735 server_base.cc:1034] running on GCE node
I20260806 09:02:01.167178 14735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:01.168733 14735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:01.169971 14735 hybrid_clock.cc:648] HybridClock initialized: now 1786006921169908 us; error 45 us; skew 500 ppm
I20260806 09:02:01.170352 14735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:01 dist-test-slave-jpj1 krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006921, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:01.174000 14735 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:01.175836 14735 webserver.cc:460] Webserver started at http://127.22.98.62:35107/ using document root <none> and password file <none>
I20260806 09:02:01.176615 14735 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:01.176739 14735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:01.177165 14735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:01.180626 14735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "a027fd4ed1f54bfe8c475a5204187311"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "e211ef08ca9f70b5ca77160ea82a9958"
server_key_iv: "6c3db744aa5878cd6c3c0d4bbd519f1e"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.181505 14735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "a027fd4ed1f54bfe8c475a5204187311"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "e211ef08ca9f70b5ca77160ea82a9958"
server_key_iv: "6c3db744aa5878cd6c3c0d4bbd519f1e"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.187566 14735 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 09:02:01.191460 14758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:01.193004 14735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:01.193199 14735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "a027fd4ed1f54bfe8c475a5204187311"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "e211ef08ca9f70b5ca77160ea82a9958"
server_key_iv: "6c3db744aa5878cd6c3c0d4bbd519f1e"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.193409 14735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:01.228358 14735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:01.234104 14735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:01.234411 14735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:01.254668 14735 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:46773
I20260806 09:02:01.254707 14830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:46773 every 8 connection(s)
I20260806 09:02:01.256274 14735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:01.256898 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 14735
I20260806 09:02:01.257115 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:01.257943 22920 external_mini_cluster.cc:1428] Setting key c83bc522e0b55a9fe05d3c248200b372
I20260806 09:02:01.261260 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:02:01 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:01.275331 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: Bootstrap starting.
I20260806 09:02:01.278728 14831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:01.280079 14831 log.cc:824] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:01.282791 14837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:01.260312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.22.98.62:46773)
0806 09:02:01.261179 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:01.261192 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:01.261237 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:02:01.262960 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:01.262984 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:01.263383 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:01.263781 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:01.265827 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.267247 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.269190 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.269995 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.270169 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:01.275228 (+ 5059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:01.275237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:01.275261 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:01.275300 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:01.278387 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:01.279286 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:01.279294 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:01.279301 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:01.279486 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:01.279984 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:01.279988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:01.279990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:01.280600 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:01.280973 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:01.281335 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:02:01.281754 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":384,"threads_started":1}
I20260806 09:02:01.283301 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: No bootstrap required, opened a new log
I20260806 09:02:01.291787 14831 raft_consensus.cc:348] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } }
I20260806 09:02:01.292156 14831 raft_consensus.cc:374] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:01.292263 14831 raft_consensus.cc:729] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a027fd4ed1f54bfe8c475a5204187311, State: Initialized, Role: FOLLOWER
I20260806 09:02:01.292817 14831 consensus_queue.cc:260] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } }
I20260806 09:02:01.293032 14831 raft_consensus.cc:388] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:01.293165 14831 raft_consensus.cc:482] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:01.293329 14831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:01.295751 14831 raft_consensus.cc:504] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } }
I20260806 09:02:01.296283 14831 leader_election.cc:302] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a027fd4ed1f54bfe8c475a5204187311; no voters:
I20260806 09:02:01.296916 14831 leader_election.cc:288] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:01.297055 14839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:01.297933 14839 raft_consensus.cc:686] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [term 1 LEADER]: Becoming Leader. State: Replica: a027fd4ed1f54bfe8c475a5204187311, State: Running, Role: LEADER
I20260806 09:02:01.298492 14839 consensus_queue.cc:237] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } }
I20260806 09:02:01.299414 14831 sys_catalog.cc:564] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:01.300591 14840 sys_catalog.cc:455] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a027fd4ed1f54bfe8c475a5204187311" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } } }
I20260806 09:02:01.301724 14840 sys_catalog.cc:458] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:01.302042 14841 sys_catalog.cc:455] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a027fd4ed1f54bfe8c475a5204187311. Latest consensus state: current_term: 1 leader_uuid: "a027fd4ed1f54bfe8c475a5204187311" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a027fd4ed1f54bfe8c475a5204187311" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46773 } } }
I20260806 09:02:01.302417 14841 sys_catalog.cc:458] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:01.304304 14848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:01.309518 14848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:01.319947 14848 catalog_manager.cc:1347] Generated new cluster ID: b3a0ee2229954db39b330879dca57bfa
I20260806 09:02:01.320068 14848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:01.334529 14848 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:01.336088 14848 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:01.347321 14848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: Generated new TSK 0
I20260806 09:02:01.348678 14848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:02:01.431455 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:01.796433 14864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:01.796864 14864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:01.797034 14864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:01.805120 14864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:01.805241 14864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:01.805394 14864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:01.815697 14864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:01.817067 14864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:01.818462 14864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:01.832448 14870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:01.832396 14874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:01.832419 14871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:01.833230 14864 server_base.cc:1034] running on GCE node
I20260806 09:02:01.833941 14864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:01.834997 14864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:01.836220 14864 hybrid_clock.cc:648] HybridClock initialized: now 1786006921836161 us; error 48 us; skew 500 ppm
I20260806 09:02:01.836560 14864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:01 dist-test-slave-jpj1 krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006921, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:01.839968 14864 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:02:01.841548 14864 webserver.cc:460] Webserver started at http://127.22.98.1:37147/ using document root <none> and password file <none>
I20260806 09:02:01.842267 14864 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:01.842379 14864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:01.842692 14864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:01.845865 14864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "62edb8cc9a5641729c5d928b3607f6df"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "81036926ef29d0424c2ba00a1d235f25"
server_key_iv: "a04f5f91327530772e1e815907664bd3"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.846815 14864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "62edb8cc9a5641729c5d928b3607f6df"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "81036926ef29d0424c2ba00a1d235f25"
server_key_iv: "a04f5f91327530772e1e815907664bd3"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.852980 14864 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 09:02:01.857131 14883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:01.859066 14864 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 09:02:01.859256 14864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "62edb8cc9a5641729c5d928b3607f6df"
format_stamp: "Formatted at 2026-08-06 09:02:01 on dist-test-slave-jpj1"
server_key: "81036926ef29d0424c2ba00a1d235f25"
server_key_iv: "a04f5f91327530772e1e815907664bd3"
server_key_version: "encryptionkey@0"
I20260806 09:02:01.859488 14864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:01.878072 14864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:01.882694 14864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:01.883018 14864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:01.884058 14864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:01.885569 14864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:01.885648 14864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:01.885751 14864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:01.885797 14864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:01.918828 14864 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:43223
I20260806 09:02:01.918874 15044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:43223 every 8 connection(s)
I20260806 09:02:01.920231 14864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:01.924005 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 14864
I20260806 09:02:01.924296 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:01.924983 22920 external_mini_cluster.cc:1428] Setting key ab29430cc503fa6866018a203709750f
Aug 06 09:02:01 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006921, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:01.942688 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:01.923061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:60441 (local address 127.22.98.62:46773)
0806 09:02:01.923578 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:01.923585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:01.924618 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 09:02:01.926953 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:01.926959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:01.927043 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:01.927134 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:01.928845 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.929867 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.931565 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.932113 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.932244 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:01.935957 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:01.935964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:01.935969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:01.936011 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:01.938970 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:01.939857 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:01.939863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:01.939867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:01.940028 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:01.940691 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:01.940697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:01.940699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:01.941010 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:01.941149 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:01.942202 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0806 09:02:01.942418 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260806 09:02:01.944113 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:01.923570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:46773 (local address 127.22.98.1:60441)
0806 09:02:01.924426 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:01.924476 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:01.926599 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:01.927422 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:01.927431 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:01.928036 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:01.928646 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:01.928665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.930056 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.930061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:01.931381 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:01.931395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:01.932371 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:01.932378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:01.932595 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:01.933324 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:01.933354 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:01.935686 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:01.939137 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:01.939146 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:01.939162 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:01.939716 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:01.940139 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:01.940144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:01.940147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:01.940432 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:01.941185 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:01.941194 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:01.941721 (+ 527us) client_negotiation.cc:769] Sending connection context
0806 09:02:01.942068 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 09:02:01.942654 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":341,"threads_started":1}
I20260806 09:02:01.945878 15045 heartbeater.cc:347] Connected to a master server at 127.22.98.62:46773
I20260806 09:02:01.946193 15045 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:01.946967 15045 heartbeater.cc:511] Master 127.22.98.62:46773 requested a full tablet report, sending...
I20260806 09:02:01.949633 14783 ts_manager.cc:194] Registered new tserver with Master: 62edb8cc9a5641729c5d928b3607f6df (127.22.98.1:43223)
I20260806 09:02:01.951718 14783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:60441
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 09:02:01.998652 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:02.372174 15053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:02.372555 15053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:02.372702 15053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:02.380316 15053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:02.380465 15053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:02.380626 15053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:02.390645 15053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:02.392064 15053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:02.393411 15053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:02.407166 15058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:02.407236 15059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:02.407194 15061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:02.408478 15053 server_base.cc:1034] running on GCE node
I20260806 09:02:02.409255 15053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:02.410248 15053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:02.411486 15053 hybrid_clock.cc:648] HybridClock initialized: now 1786006922411422 us; error 46 us; skew 500 ppm
I20260806 09:02:02.411914 15053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:02 dist-test-slave-jpj1 krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006922, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:02.415156 15053 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:02:02.416842 15053 webserver.cc:460] Webserver started at http://127.22.98.2:41781/ using document root <none> and password file <none>
I20260806 09:02:02.417568 15053 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:02.417688 15053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:02.417995 15053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:02.421205 15053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "7ba7846f75e345368b51c54e5fde72f7"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "6a43216d38a3bfdae6e4aebcbded7940"
server_key_iv: "9d389a6a1dc96e893b961cb7e35fd361"
server_key_version: "encryptionkey@0"
I20260806 09:02:02.422071 15053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "7ba7846f75e345368b51c54e5fde72f7"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "6a43216d38a3bfdae6e4aebcbded7940"
server_key_iv: "9d389a6a1dc96e893b961cb7e35fd361"
server_key_version: "encryptionkey@0"
I20260806 09:02:02.428143 15053 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 09:02:02.432210 15072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:02.433959 15053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:02.434158 15053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "7ba7846f75e345368b51c54e5fde72f7"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "6a43216d38a3bfdae6e4aebcbded7940"
server_key_iv: "9d389a6a1dc96e893b961cb7e35fd361"
server_key_version: "encryptionkey@0"
I20260806 09:02:02.434358 15053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:02.460608 15053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:02.465750 15053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:02.466033 15053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:02.467105 15053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:02.468622 15053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:02.468703 15053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:02.468804 15053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:02.468851 15053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:02.503229 15053 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:45079
I20260806 09:02:02.503307 15230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:45079 every 8 connection(s)
I20260806 09:02:02.504594 15053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:02.513283 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15053
I20260806 09:02:02.513458 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:02.513947 22920 external_mini_cluster.cc:1428] Setting key 40690b47128995f0ccce849697c7536a
Aug 06 09:02:02 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006922, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:02.529027 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:02.507350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:51841 (local address 127.22.98.62:46773)
0806 09:02:02.507891 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:02.507897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:02.509011 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 09:02:02.510979 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:02.510986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:02.511075 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:02.511172 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:02.513021 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:02.513915 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:02.515963 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:02.516571 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:02.516679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:02.521593 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:02.521600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:02.521604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:02.521650 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:02.525134 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:02.526145 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:02.526152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:02.526155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:02.526348 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:02.526959 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:02.526964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:02.526966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:02.527346 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:02.527492 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:02.528400 (+ 908us) server_negotiation.cc:299] Negotiation successful
0806 09:02:02.528657 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":274,"threads_started":1}
I20260806 09:02:02.529970 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:02.507926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:46773 (local address 127.22.98.2:51841)
0806 09:02:02.508792 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:02.508852 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:02.510672 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:02.511469 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:02.511478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:02.512100 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:02.512765 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:02.512786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:02.514105 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:02.514111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:02.515807 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:02.515823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:02.516852 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:02.516863 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:02.517182 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:02.517996 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:02.518030 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:02.521238 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:02.525362 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:02.525373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:02.525391 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:02.525968 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:02.526511 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:02.526515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:02.526518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:02.526822 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:02.527497 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:02.527504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:02.527986 (+ 482us) client_negotiation.cc:769] Sending connection context
0806 09:02:02.528251 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 09:02:02.528708 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":331,"threads_started":1}
I20260806 09:02:02.531646 15231 heartbeater.cc:347] Connected to a master server at 127.22.98.62:46773
I20260806 09:02:02.531975 15231 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:02.532730 15231 heartbeater.cc:511] Master 127.22.98.62:46773 requested a full tablet report, sending...
I20260806 09:02:02.534480 14783 ts_manager.cc:194] Registered new tserver with Master: 7ba7846f75e345368b51c54e5fde72f7 (127.22.98.2:45079)
I20260806 09:02:02.535689 14783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:51841
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:02:02.587990 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:02.945729 15238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:02.946063 15238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:02.946210 15238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:02.953759 15238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:02.953908 15238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:02.954066 15238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:02.955361 15045 heartbeater.cc:503] Master 127.22.98.62:46773 was elected leader, sending a full tablet report...
I20260806 09:02:02.964761 15238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:46773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:02.966107 15238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:02.967422 15238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:02.980988 15250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:02.981014 15247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:02.981060 15248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:02.981989 15238 server_base.cc:1034] running on GCE node
I20260806 09:02:02.982720 15238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:02.983790 15238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:02.985045 15238 hybrid_clock.cc:648] HybridClock initialized: now 1786006922984979 us; error 53 us; skew 500 ppm
I20260806 09:02:02.985414 15238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:02 dist-test-slave-jpj1 krb5kdc[14716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006922, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:02.988924 15238 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:02:02.990880 15238 webserver.cc:460] Webserver started at http://127.22.98.3:39857/ using document root <none> and password file <none>
I20260806 09:02:02.991823 15238 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:02.991971 15238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:02.992313 15238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:02.995529 15238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "e2b16e4ef9544b4aa7c4336fe471788f"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "90d9614a76e1a3cc10623d6242e20ba4"
server_key_iv: "8bd781b479be72f577e4f5dcae180488"
server_key_version: "encryptionkey@0"
I20260806 09:02:02.996413 15238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "e2b16e4ef9544b4aa7c4336fe471788f"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "90d9614a76e1a3cc10623d6242e20ba4"
server_key_iv: "8bd781b479be72f577e4f5dcae180488"
server_key_version: "encryptionkey@0"
I20260806 09:02:03.002995 15238 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 09:02:03.010994 15257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:03.012619 15238 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.003s
I20260806 09:02:03.012787 15238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "e2b16e4ef9544b4aa7c4336fe471788f"
format_stamp: "Formatted at 2026-08-06 09:02:02 on dist-test-slave-jpj1"
server_key: "90d9614a76e1a3cc10623d6242e20ba4"
server_key_iv: "8bd781b479be72f577e4f5dcae180488"
server_key_version: "encryptionkey@0"
I20260806 09:02:03.012986 15238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:03.046278 15238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:03.050899 15238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:03.051167 15238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:03.052198 15238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:03.053574 15238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:03.053663 15238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:03.053761 15238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:03.053804 15238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:03.087294 15238 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:44113
I20260806 09:02:03.087388 15415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:44113 every 8 connection(s)
I20260806 09:02:03.088711 15238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:03.091488 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15238
I20260806 09:02:03.091714 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:03.092299 22920 external_mini_cluster.cc:1428] Setting key baf34b605ccb89e63a48174868c8218e
Aug 06 09:02:03 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006922, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:03.111384 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.091574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:57525 (local address 127.22.98.62:46773)
0806 09:02:03.092128 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.092135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.093200 (+ 1065us) server_negotiation.cc:407] Connection header received
0806 09:02:03.095659 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.095664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.095757 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.095884 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:03.097815 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.098743 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.100671 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.101091 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.101206 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.105181 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:03.105187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:03.105191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:03.105236 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:03.108231 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.109061 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.109066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.109068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.109211 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.109708 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.109712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.109715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.110007 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:03.110142 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.110883 (+ 741us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.111073 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":247,"threads_started":1}
I20260806 09:02:03.112380 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.092097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:46773 (local address 127.22.98.3:57525)
0806 09:02:03.093016 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.093060 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.095308 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.096155 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.096169 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.096847 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:03.097534 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.097555 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.098940 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.098944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.100493 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.100510 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.101388 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.101396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.101624 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.102439 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:03.102483 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:03.104902 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:03.108410 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:03.108421 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:03.108442 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:03.108905 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:03.109300 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:03.109304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:03.109306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:03.109543 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:03.110114 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:03.110120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:03.110501 (+ 381us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.110756 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.111188 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":381,"threads_started":1}
I20260806 09:02:03.113941 15417 heartbeater.cc:347] Connected to a master server at 127.22.98.62:46773
I20260806 09:02:03.114260 15417 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:03.114991 15417 heartbeater.cc:511] Master 127.22.98.62:46773 requested a full tablet report, sending...
I20260806 09:02:03.116534 14783 ts_manager.cc:194] Registered new tserver with Master: e2b16e4ef9544b4aa7c4336fe471788f (127.22.98.3:44113)
I20260806 09:02:03.117511 14783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:57525
I20260806 09:02:03.121320 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:03.140522 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.126166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.22.98.62:46773)
0806 09:02:03.126350 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.126354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.126767 (+ 413us) server_negotiation.cc:407] Connection header received
0806 09:02:03.126853 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.126858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.126928 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.127019 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:03.128752 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.129564 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.131139 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.131649 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.131762 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.134348 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:03.134354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:03.134358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:03.134396 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:03.137361 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.138232 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.138237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.138240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.138393 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.138885 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.138890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.138892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.139369 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:03.139666 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.140062 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.140277 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:02:03.145072 14783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:03.152593 14783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:03.184998 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.167486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.22.98.2:45079)
0806 09:02:03.172291 (+ 4805us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.172304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.172341 (+ 37us) server_negotiation.cc:407] Connection header received
0806 09:02:03.172461 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.172610 (+ 149us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.172964 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.177118 (+ 4154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:03.177477 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.179353 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.182701 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.183776 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.183917 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.184163 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.184503 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.184688 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4601,"thread_start_us":4459,"threads_started":1}
I20260806 09:02:03.185073 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.166741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:45079 (local address 127.0.0.1:40104)
0806 09:02:03.171552 (+ 4811us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.171583 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.171714 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.173264 (+ 1550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.173274 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.173332 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:03.173839 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.173854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.179557 (+ 5703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.179564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.182466 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.182485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.184012 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.184019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.184233 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.184280 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.184396 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.184618 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4636,"mutex_wait_us":54,"thread_start_us":1620,"threads_started":1}
I20260806 09:02:03.187935 15435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.166738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.22.98.1:43223)
0806 09:02:03.179339 (+ 12601us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.179347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.179374 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:02:03.179472 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.179479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.179749 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.180006 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:03.180985 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.182228 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.184930 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.186765 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.187009 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.187253 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.187425 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.187630 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12414,"thread_start_us":12235,"threads_started":1}
I20260806 09:02:03.187978 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.166538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43223 (local address 127.0.0.1:49302)
0806 09:02:03.168921 (+ 2383us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.169001 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.169249 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.180007 (+ 10758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.180012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.180049 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:03.180751 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.180765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.182383 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.182388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.184692 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.184709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.187040 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.187048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.187238 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.187272 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.187367 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.187515 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1972,"mutex_wait_us":120,"thread_start_us":4650,"threads_started":1}
I20260806 09:02:03.189129 15440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.172410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:44113 (local address 127.0.0.1:48552)
0806 09:02:03.177727 (+ 5317us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.177760 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.177982 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.179643 (+ 1661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.179648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.179681 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:03.180507 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.180525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.182600 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.182606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.184836 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.184849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.188307 (+ 3458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.188317 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.188490 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.188523 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.188623 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.188760 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5143,"mutex_wait_us":4,"thread_start_us":5685,"threads_started":1}
I20260806 09:02:03.189260 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.172126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.22.98.3:44113)
0806 09:02:03.178882 (+ 6756us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.178892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.178936 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:02:03.179059 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.179069 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.179379 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.179668 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:03.180860 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.182444 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.185011 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.186451 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.186587 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.186844 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.188728 (+ 1884us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.188913 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6566,"thread_start_us":6394,"threads_started":1}
I20260806 09:02:03.189806 15139 tablet_service.cc:1467] Processing CreateTablet for tablet 05b7316e1bca4767957d934c03ca93f3 (DEFAULT_TABLE table=test-table [id=eba8bcf786ba4023be40f0ae417a99c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:03.191162 15139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b7316e1bca4767957d934c03ca93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:03.191721 14953 tablet_service.cc:1467] Processing CreateTablet for tablet 05b7316e1bca4767957d934c03ca93f3 (DEFAULT_TABLE table=test-table [id=eba8bcf786ba4023be40f0ae417a99c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:03.193338 14953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b7316e1bca4767957d934c03ca93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:03.195571 15326 tablet_service.cc:1467] Processing CreateTablet for tablet 05b7316e1bca4767957d934c03ca93f3 (DEFAULT_TABLE table=test-table [id=eba8bcf786ba4023be40f0ae417a99c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:03.197403 15326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05b7316e1bca4767957d934c03ca93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:03.204396 15442 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Bootstrap starting.
I20260806 09:02:03.207998 15442 tablet_bootstrap.cc:654] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:03.209646 15442 log.cc:824] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:03.212411 15443 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Bootstrap starting.
I20260806 09:02:03.216406 15445 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Bootstrap starting.
I20260806 09:02:03.218130 15443 tablet_bootstrap.cc:654] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:03.219583 15443 log.cc:824] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:03.219894 15445 tablet_bootstrap.cc:654] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:03.220047 15442 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: No bootstrap required, opened a new log
I20260806 09:02:03.220432 15442 ts_tablet_manager.cc:1397] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.004s
I20260806 09:02:03.221251 15445 log.cc:824] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:03.224444 15443 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: No bootstrap required, opened a new log
I20260806 09:02:03.224726 15443 ts_tablet_manager.cc:1397] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260806 09:02:03.224756 15445 tablet_bootstrap.cc:492] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: No bootstrap required, opened a new log
I20260806 09:02:03.225056 15445 ts_tablet_manager.cc:1397] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260806 09:02:03.231647 15442 raft_consensus.cc:348] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.232718 15442 raft_consensus.cc:374] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:03.232944 15442 raft_consensus.cc:729] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba7846f75e345368b51c54e5fde72f7, State: Initialized, Role: FOLLOWER
I20260806 09:02:03.232764 15443 raft_consensus.cc:348] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.233291 15443 raft_consensus.cc:374] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:03.233412 15443 raft_consensus.cc:729] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62edb8cc9a5641729c5d928b3607f6df, State: Initialized, Role: FOLLOWER
I20260806 09:02:03.233870 15442 consensus_queue.cc:260] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.234273 15443 consensus_queue.cc:260] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.236099 15445 raft_consensus.cc:348] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.236505 15231 heartbeater.cc:503] Master 127.22.98.62:46773 was elected leader, sending a full tablet report...
I20260806 09:02:03.236601 15445 raft_consensus.cc:374] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:03.236721 15445 raft_consensus.cc:729] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b16e4ef9544b4aa7c4336fe471788f, State: Initialized, Role: FOLLOWER
I20260806 09:02:03.237435 15442 ts_tablet_manager.cc:1428] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260806 09:02:03.237442 15445 consensus_queue.cc:260] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.240358 15443 ts_tablet_manager.cc:1428] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260806 09:02:03.241747 15417 heartbeater.cc:503] Master 127.22.98.62:46773 was elected leader, sending a full tablet report...
I20260806 09:02:03.242467 15445 ts_tablet_manager.cc:1428] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Time spent starting tablet: real 0.017s user 0.010s sys 0.007s
W20260806 09:02:03.259982 15232 tablet.cc:2369] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:03.340560 15421 tablet.cc:2369] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:03.423007 15046 tablet.cc:2369] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:03.440418 15448 raft_consensus.cc:482] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:03.440737 15448 raft_consensus.cc:504] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.442466 15448 leader_election.cc:288] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62edb8cc9a5641729c5d928b3607f6df (127.22.98.1:43223), e2b16e4ef9544b4aa7c4336fe471788f (127.22.98.3:44113)
I20260806 09:02:03.449347 15435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.442888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:46435 (local address 127.22.98.1:43223)
0806 09:02:03.443080 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.443085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.443621 (+ 536us) server_negotiation.cc:407] Connection header received
0806 09:02:03.443950 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.443956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.444040 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.444107 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:03.445000 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.445724 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.447588 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.448410 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.448540 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.448652 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.448968 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.449099 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:02:03.449478 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.442865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:43223 (local address 127.22.98.2:46435)
0806 09:02:03.443483 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.443500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.443703 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.444359 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.444365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.444396 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:03.444851 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.444864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.445857 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.445863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.447394 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.447407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.448652 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.448659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.448810 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.448835 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.448927 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.449072 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":49,"thread_start_us":344,"threads_started":1}
I20260806 09:02:03.449687 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.443815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:58617 (local address 127.22.98.3:44113)
0806 09:02:03.444031 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.444038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.444645 (+ 607us) server_negotiation.cc:407] Connection header received
0806 09:02:03.444859 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.444863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.444923 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.444995 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:03.445540 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.446235 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.447957 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.448768 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.448883 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.448983 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.449288 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.449414 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:02:03.450008 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.443649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:44113 (local address 127.22.98.2:58617)
0806 09:02:03.444524 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:03.444544 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:03.444699 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:03.445026 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:03.445030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:03.445054 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:03.445370 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.445378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.446371 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.446377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.447813 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:03.447826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.448981 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.448987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:03.449133 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.449155 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:02:03.449252 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 09:02:03.449379 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":2,"thread_start_us":478,"threads_started":1}
I20260806 09:02:03.450666 14983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b7316e1bca4767957d934c03ca93f3" candidate_uuid: "7ba7846f75e345368b51c54e5fde72f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62edb8cc9a5641729c5d928b3607f6df" is_pre_election: true
I20260806 09:02:03.451004 15352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b7316e1bca4767957d934c03ca93f3" candidate_uuid: "7ba7846f75e345368b51c54e5fde72f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" is_pre_election: true
I20260806 09:02:03.451195 14983 raft_consensus.cc:2454] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba7846f75e345368b51c54e5fde72f7 in term 0.
I20260806 09:02:03.451493 15352 raft_consensus.cc:2454] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba7846f75e345368b51c54e5fde72f7 in term 0.
I20260806 09:02:03.451993 15073 leader_election.cc:302] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62edb8cc9a5641729c5d928b3607f6df, 7ba7846f75e345368b51c54e5fde72f7; no voters:
I20260806 09:02:03.452381 15448 raft_consensus.cc:2781] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:03.452514 15448 raft_consensus.cc:482] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:03.452622 15448 raft_consensus.cc:3037] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:03.454846 15448 raft_consensus.cc:504] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.455447 15448 leader_election.cc:288] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [CANDIDATE]: Term 1 election: Requested vote from peers 62edb8cc9a5641729c5d928b3607f6df (127.22.98.1:43223), e2b16e4ef9544b4aa7c4336fe471788f (127.22.98.3:44113)
I20260806 09:02:03.456045 14983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b7316e1bca4767957d934c03ca93f3" candidate_uuid: "7ba7846f75e345368b51c54e5fde72f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62edb8cc9a5641729c5d928b3607f6df"
I20260806 09:02:03.456140 15352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05b7316e1bca4767957d934c03ca93f3" candidate_uuid: "7ba7846f75e345368b51c54e5fde72f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b16e4ef9544b4aa7c4336fe471788f"
I20260806 09:02:03.456238 14983 raft_consensus.cc:3037] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:03.456364 15352 raft_consensus.cc:3037] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:03.458573 14983 raft_consensus.cc:2454] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba7846f75e345368b51c54e5fde72f7 in term 1.
I20260806 09:02:03.458611 15352 raft_consensus.cc:2454] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba7846f75e345368b51c54e5fde72f7 in term 1.
I20260806 09:02:03.459014 15073 leader_election.cc:302] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62edb8cc9a5641729c5d928b3607f6df, 7ba7846f75e345368b51c54e5fde72f7; no voters:
I20260806 09:02:03.459318 15448 raft_consensus.cc:2781] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:03.459827 15448 raft_consensus.cc:686] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba7846f75e345368b51c54e5fde72f7, State: Running, Role: LEADER
I20260806 09:02:03.460340 15448 consensus_queue.cc:237] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } }
I20260806 09:02:03.465623 14781 catalog_manager.cc:5495] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ba7846f75e345368b51c54e5fde72f7 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "7ba7846f75e345368b51c54e5fde72f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ba7846f75e345368b51c54e5fde72f7" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 } health_report { overall_health: UNKNOWN } } }
I20260806 09:02:03.552116 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.546279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.22.98.2:45079)
0806 09:02:03.546469 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.546474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.546494 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:02:03.546625 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.546628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.546730 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.546997 (+ 267us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:03.547414 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.548178 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.549714 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.550169 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.550278 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.550910 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:03.551444 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:03.551623 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.551792 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.551872 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:02:03.561316 15352 raft_consensus.cc:1261] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 1 FOLLOWER]: Refusing update from remote peer 7ba7846f75e345368b51c54e5fde72f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:03.561311 14983 raft_consensus.cc:1261] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 1 FOLLOWER]: Refusing update from remote peer 7ba7846f75e345368b51c54e5fde72f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:03.562282 15456 consensus_queue.cc:1035] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62edb8cc9a5641729c5d928b3607f6df" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:03.562705 15448 consensus_queue.cc:1035] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b16e4ef9544b4aa7c4336fe471788f" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 44113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:03.576668 15460 mvcc.cc:205] Tried to move back new op lower bound from 7315484358895669248 to 7315484358497808384. Current Snapshot: MvccSnapshot[applied={T|T < 7315484358895669248}]
I20260806 09:02:03.579412 15462 mvcc.cc:205] Tried to move back new op lower bound from 7315484358895669248 to 7315484358497808384. Current Snapshot: MvccSnapshot[applied={T|T < 7315484358895669248}]
I20260806 09:02:03.590927 14781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:03.591202 14781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:03.595101 14781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a027fd4ed1f54bfe8c475a5204187311: Sending DeleteTablet for 3 replicas of tablet 05b7316e1bca4767957d934c03ca93f3
I20260806 09:02:03.596467 15139 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b7316e1bca4767957d934c03ca93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:40104
I20260806 09:02:03.596463 14953 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b7316e1bca4767957d934c03ca93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:49302
I20260806 09:02:03.597818 15326 tablet_service.cc:1514] Processing DeleteTablet for tablet 05b7316e1bca4767957d934c03ca93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:03 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:48552
I20260806 09:02:03.598084 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 14864
I20260806 09:02:03.602051 15474 tablet_replica.cc:330] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: stopping tablet replica
I20260806 09:02:03.602543 15474 raft_consensus.cc:2229] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:03.603142 15474 raft_consensus.cc:2258] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:03.604139 15472 tablet_replica.cc:330] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: stopping tablet replica
I20260806 09:02:03.604560 15472 raft_consensus.cc:2229] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:03.605283 15472 raft_consensus.cc:2258] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:03.606020 15474 ts_tablet_manager.cc:1905] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:03.607651 15472 ts_tablet_manager.cc:1905] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:03.614074 15474 ts_tablet_manager.cc:1918] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:03.614271 15474 log.cc:1196] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/05b7316e1bca4767957d934c03ca93f3
I20260806 09:02:03.614842 15474 ts_tablet_manager.cc:1939] T 05b7316e1bca4767957d934c03ca93f3 P e2b16e4ef9544b4aa7c4336fe471788f: Deleting consensus metadata
I20260806 09:02:03.615144 15472 ts_tablet_manager.cc:1918] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:03.615314 15472 log.cc:1196] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/05b7316e1bca4767957d934c03ca93f3
I20260806 09:02:03.615932 15472 ts_tablet_manager.cc:1939] T 05b7316e1bca4767957d934c03ca93f3 P 7ba7846f75e345368b51c54e5fde72f7: Deleting consensus metadata
I20260806 09:02:03.616850 15473 tablet_replica.cc:330] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: stopping tablet replica
I20260806 09:02:03.617271 15473 raft_consensus.cc:2229] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:03.617910 15473 raft_consensus.cc:2258] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:03.617835 14762 catalog_manager.cc:4841] TS 7ba7846f75e345368b51c54e5fde72f7 (127.22.98.2:45079): tablet 05b7316e1bca4767957d934c03ca93f3 (table test-table [id=eba8bcf786ba4023be40f0ae417a99c8]) successfully deleted
I20260806 09:02:03.620057 15473 ts_tablet_manager.cc:1905] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:03.620173 14760 catalog_manager.cc:4841] TS e2b16e4ef9544b4aa7c4336fe471788f (127.22.98.3:44113): tablet 05b7316e1bca4767957d934c03ca93f3 (table test-table [id=eba8bcf786ba4023be40f0ae417a99c8]) successfully deleted
Aug 06 09:02:03 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:02:03.626464 15473 ts_tablet_manager.cc:1918] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:03.626662 15473 log.cc:1196] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/05b7316e1bca4767957d934c03ca93f3
I20260806 09:02:03.627236 15473 ts_tablet_manager.cc:1939] T 05b7316e1bca4767957d934c03ca93f3 P 62edb8cc9a5641729c5d928b3607f6df: Deleting consensus metadata
I20260806 09:02:03.629354 14760 catalog_manager.cc:4841] TS 62edb8cc9a5641729c5d928b3607f6df (127.22.98.1:43223): tablet 05b7316e1bca4767957d934c03ca93f3 (table test-table [id=eba8bcf786ba4023be40f0ae417a99c8]) successfully deleted
I20260806 09:02:03.632916 15435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.599466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.22.98.1:43223)
0806 09:02:03.599714 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.599722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.603233 (+ 3511us) server_negotiation.cc:407] Connection header received
0806 09:02:03.613289 (+ 10056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.613296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.613443 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.613535 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:03.615649 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.616909 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.618975 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.619654 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.619787 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.625718 (+ 5931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:03.625734 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:03.625756 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:03.625810 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:03.629754 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.630764 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.630772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.630774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.630930 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.631359 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.631362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.631364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.631809 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:03.632034 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.632425 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.632597 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 09:02:03.633592 15036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:03.741541 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 14864
I20260806 09:02:03.765942 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15053
Aug 06 09:02:03 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:02:03.784515 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.766773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.22.98.2:45079)
0806 09:02:03.766974 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.766980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.767123 (+ 143us) server_negotiation.cc:407] Connection header received
0806 09:02:03.767351 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.767355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.767421 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.767503 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:03.769238 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.770072 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.772042 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.772528 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.772629 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.777885 (+ 5256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:03.777900 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:03.777918 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:03.777965 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:03.781391 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.782228 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.782240 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.782243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.782403 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.782815 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.782819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.782821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.783212 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:03.783633 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.783899 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.784148 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:02:03.785324 15224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:03.900098 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15053
I20260806 09:02:03.925180 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15238
Aug 06 09:02:03 dist-test-slave-jpj1 krb5kdc[14716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:02:03.945019 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:03.925992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.22.98.3:44113)
0806 09:02:03.926182 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:03.926188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:03.926210 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:02:03.926445 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:03.926450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:03.926541 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:03.926630 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:03.928018 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.928747 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.930156 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:03.930552 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:03.930665 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:03.939041 (+ 8376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:03.939060 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:03.939080 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:03.939122 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:03.942011 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.942773 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.942784 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.942786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.942945 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:03.943346 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:03.943350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:03.943352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:03.943786 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:03.944078 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:03.944417 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 09:02:03.944651 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:02:03.945712 15410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:04.035836 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15238
I20260806 09:02:04.057024 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 14735
I20260806 09:02:04.058221 14824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:04.171737 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 14735
2026-08-06T09:02:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4954 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T09:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:04Z Disabled control of system clock
I20260806 09:02:04.275297 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41229
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:04.639879 15483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:04.640189 15483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:04.640287 15483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:04.647703 15483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:04.647830 15483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:04.647868 15483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:04.647928 15483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:04.647972 15483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:04.657603 15483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:41229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:04.658967 15483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:04.660923 15483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:04.673612 15490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:04.673673 15491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:04.673560 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:04.674572 15483 server_base.cc:1034] running on GCE node
I20260806 09:02:04.675475 15483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:04.677073 15483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:04.678303 15483 hybrid_clock.cc:648] HybridClock initialized: now 1786006924678236 us; error 53 us; skew 500 ppm
I20260806 09:02:04.678683 15483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:04.681473 15483 webserver.cc:460] Webserver started at http://127.22.98.62:46349/ using document root <none> and password file <none>
I20260806 09:02:04.682324 15483 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:04.682443 15483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:04.682790 15483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:04.686018 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "9fa84f517d1c4bef80ca0ebab65f3e77"
format_stamp: "Formatted at 2026-08-06 09:02:04 on dist-test-slave-jpj1"
server_key: "fffdcdc6e89b6739f856b667282f1b22"
server_key_iv: "93689984c2994d40592b16ce26b76f4e"
server_key_version: "encryptionkey@0"
I20260806 09:02:04.686878 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "9fa84f517d1c4bef80ca0ebab65f3e77"
format_stamp: "Formatted at 2026-08-06 09:02:04 on dist-test-slave-jpj1"
server_key: "fffdcdc6e89b6739f856b667282f1b22"
server_key_iv: "93689984c2994d40592b16ce26b76f4e"
server_key_version: "encryptionkey@0"
I20260806 09:02:04.692646 15483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 09:02:04.696241 15504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:04.697654 15483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:04.697844 15483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "9fa84f517d1c4bef80ca0ebab65f3e77"
format_stamp: "Formatted at 2026-08-06 09:02:04 on dist-test-slave-jpj1"
server_key: "fffdcdc6e89b6739f856b667282f1b22"
server_key_iv: "93689984c2994d40592b16ce26b76f4e"
server_key_version: "encryptionkey@0"
I20260806 09:02:04.698055 15483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:04.719664 15483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:04.720587 15483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:04.720878 15483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:04.739131 15483 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:41229
I20260806 09:02:04.739172 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:41229 every 8 connection(s)
I20260806 09:02:04.740501 15483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:04.744426 15587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:04.747033 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15483
I20260806 09:02:04.747205 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:04.747812 22920 external_mini_cluster.cc:1428] Setting key d5d7e7ecc2b14d13d27c9c4d02053108
I20260806 09:02:04.756553 15587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: Bootstrap starting.
I20260806 09:02:04.759966 15587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:04.761173 15587 log.cc:824] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:04.761716 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:04.749881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.22.98.62:41229)
0806 09:02:04.750953 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:04.750964 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:04.751009 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:02:04.752771 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:04.752795 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:04.753269 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:04.753676 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:04.754298 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:04.755632 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:04.757072 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:04.757765 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:04.757953 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:04.759226 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:04.759236 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:04.759250 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:04.759874 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:04.759975 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:04.760009 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:04.760116 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:04.760362 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 09:02:04.760683 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":384,"threads_started":1}
I20260806 09:02:04.764446 15587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: No bootstrap required, opened a new log
I20260806 09:02:04.772321 15587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } }
I20260806 09:02:04.772681 15587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:04.772760 15587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa84f517d1c4bef80ca0ebab65f3e77, State: Initialized, Role: FOLLOWER
I20260806 09:02:04.773304 15587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } }
I20260806 09:02:04.773520 15587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:04.773623 15587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:04.773749 15587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:04.776014 15587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } }
I20260806 09:02:04.776638 15587 leader_election.cc:302] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fa84f517d1c4bef80ca0ebab65f3e77; no voters:
I20260806 09:02:04.777184 15587 leader_election.cc:288] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:04.777338 15593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:04.778000 15593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [term 1 LEADER]: Becoming Leader. State: Replica: 9fa84f517d1c4bef80ca0ebab65f3e77, State: Running, Role: LEADER
I20260806 09:02:04.778522 15593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } }
I20260806 09:02:04.779208 15587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:04.781527 15596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fa84f517d1c4bef80ca0ebab65f3e77. Latest consensus state: current_term: 1 leader_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } } }
I20260806 09:02:04.781659 15594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fa84f517d1c4bef80ca0ebab65f3e77" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 41229 } } }
I20260806 09:02:04.781930 15596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:04.781996 15594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:04.783419 15604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:04.791378 15604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:04.801811 15604 catalog_manager.cc:1347] Generated new cluster ID: 986bc01f64d94404b39faae0fd2b078a
I20260806 09:02:04.801942 15604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:04.816591 15604 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:04.818008 15604 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:04.830929 15604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: Generated new TSK 0
I20260806 09:02:04.832257 15604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:04.841945 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--builtin_ntp_servers=127.22.98.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:05.201939 15616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:05.202244 15616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:05.202380 15616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:05.210103 15616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:05.210217 15616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:05.210361 15616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:05.220009 15616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:05.221508 15616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:05.223584 15616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:05.238826 15627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:05.238879 15625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:05.238907 15624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:05.239866 15616 server_base.cc:1034] running on GCE node
I20260806 09:02:05.240532 15616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:05.241585 15616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:05.242877 15616 hybrid_clock.cc:648] HybridClock initialized: now 1786006925242800 us; error 55 us; skew 500 ppm
I20260806 09:02:05.243292 15616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:05.246248 15616 webserver.cc:460] Webserver started at http://127.22.98.1:41479/ using document root <none> and password file <none>
I20260806 09:02:05.247067 15616 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:05.247175 15616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:05.247526 15616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:05.250742 15616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "f8484947393d4f7394be5406bc907422"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "603eff906fdb1227add927dad6840787"
server_key_iv: "8982594d40840118982ae0a918c9085f"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.251670 15616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "f8484947393d4f7394be5406bc907422"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "603eff906fdb1227add927dad6840787"
server_key_iv: "8982594d40840118982ae0a918c9085f"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.257395 15616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:02:05.261399 15634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.262895 15616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 09:02:05.263043 15616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "f8484947393d4f7394be5406bc907422"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "603eff906fdb1227add927dad6840787"
server_key_iv: "8982594d40840118982ae0a918c9085f"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.263247 15616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:05.281827 15616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:05.282717 15616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:05.283093 15616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:05.284349 15616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:05.285956 15616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:05.286054 15616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.286185 15616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:05.286264 15616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.319911 15616 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:45115
I20260806 09:02:05.319970 15797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:45115 every 8 connection(s)
I20260806 09:02:05.321290 15616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:05.326278 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15616
I20260806 09:02:05.326442 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:05.326869 22920 external_mini_cluster.cc:1428] Setting key 4a14d5ba45f1380d87f30df0fcae2dad
I20260806 09:02:05.329643 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--builtin_ntp_servers=127.22.98.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:05.334846 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:05.323986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:33137 (local address 127.22.98.62:41229)
0806 09:02:05.324490 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:05.324494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:05.325776 (+ 1282us) server_negotiation.cc:407] Connection header received
0806 09:02:05.327842 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:05.327848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:05.327936 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:05.328036 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:05.329275 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.330183 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.331947 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.332361 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.332475 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:05.333940 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:05.333946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:05.333949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:05.333969 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:05.334023 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:05.334040 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:05.334122 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:05.334356 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 09:02:05.334466 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":266,"threads_started":1}
I20260806 09:02:05.336005 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:05.324609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41229 (local address 127.22.98.1:33137)
0806 09:02:05.325603 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:05.325644 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:05.327509 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:05.328297 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:05.328306 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:05.328400 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:05.329099 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:05.329118 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.330352 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.330356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:05.331798 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:05.331812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.332601 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.332609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:05.332848 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:05.333574 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:05.333577 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:05.333621 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:05.333707 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:05.334138 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:05.334150 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:05.334158 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 09:02:05.334332 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 09:02:05.334803 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":304,"threads_started":1}
I20260806 09:02:05.337775 15798 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41229
I20260806 09:02:05.338101 15798 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:05.338907 15798 heartbeater.cc:511] Master 127.22.98.62:41229 requested a full tablet report, sending...
I20260806 09:02:05.341364 15531 ts_manager.cc:194] Registered new tserver with Master: f8484947393d4f7394be5406bc907422 (127.22.98.1:45115)
I20260806 09:02:05.343053 15531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.1:33137
W20260806 09:02:05.686276 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:05.686600 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:05.686746 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:05.694077 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:05.694206 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:05.694355 15803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:05.704068 15803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:05.705444 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:05.707348 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:05.720747 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:05.720794 15811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:05.720815 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:05.722075 15803 server_base.cc:1034] running on GCE node
I20260806 09:02:05.722708 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:05.723723 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:05.724956 15803 hybrid_clock.cc:648] HybridClock initialized: now 1786006925724899 us; error 43 us; skew 500 ppm
I20260806 09:02:05.725310 15803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:05.727730 15803 webserver.cc:460] Webserver started at http://127.22.98.2:46169/ using document root <none> and password file <none>
I20260806 09:02:05.728438 15803 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:05.728544 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:05.728857 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:05.731882 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "ba31c2d8ffdb43d7842c643c488bb356"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "349bd5af1229824695783714587236d2"
server_key_iv: "dc315b858795da90c4300d5bfc60a0f7"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.732736 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "ba31c2d8ffdb43d7842c643c488bb356"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "349bd5af1229824695783714587236d2"
server_key_iv: "dc315b858795da90c4300d5bfc60a0f7"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.738682 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 09:02:05.742880 15817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.744477 15803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:02:05.744660 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "ba31c2d8ffdb43d7842c643c488bb356"
format_stamp: "Formatted at 2026-08-06 09:02:05 on dist-test-slave-jpj1"
server_key: "349bd5af1229824695783714587236d2"
server_key_iv: "dc315b858795da90c4300d5bfc60a0f7"
server_key_version: "encryptionkey@0"
I20260806 09:02:05.744907 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:05.766129 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:05.767071 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:05.767355 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:05.768498 15803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:05.770089 15803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:05.770182 15803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.770300 15803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:05.770370 15803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:05.802644 15803 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:35531
I20260806 09:02:05.802764 15969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:35531 every 8 connection(s)
I20260806 09:02:05.804011 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:05.813858 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15803
I20260806 09:02:05.814041 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:05.814545 22920 external_mini_cluster.cc:1428] Setting key 1eb1ff853803a86cbf521d3e72581cf8
I20260806 09:02:05.817004 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:05.806606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:49025 (local address 127.22.98.62:41229)
0806 09:02:05.806825 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:05.806832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:05.808533 (+ 1701us) server_negotiation.cc:407] Connection header received
0806 09:02:05.810404 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:05.810410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:05.810486 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:05.810585 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:05.811744 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.812499 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.814141 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.814562 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.814682 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:05.816173 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:05.816179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:05.816182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:05.816200 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:05.816233 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:05.816245 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:05.816331 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:05.816570 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 09:02:05.816674 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:02:05.817420 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--builtin_ntp_servers=127.22.98.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:05.818086 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:05.807194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41229 (local address 127.22.98.2:49025)
0806 09:02:05.808304 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:05.808368 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:05.810097 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:05.810775 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:05.810786 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:05.810878 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:05.811516 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:05.811565 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.812637 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.812640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:05.813941 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:05.813955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:05.814801 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:05.814809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:05.815041 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:05.815849 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:05.815852 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:05.815895 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:05.815984 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:05.816348 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:05.816360 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:05.816368 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 09:02:05.816525 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 09:02:05.816964 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":357,"threads_started":1}
I20260806 09:02:05.819610 15971 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41229
I20260806 09:02:05.819932 15971 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:05.820719 15971 heartbeater.cc:511] Master 127.22.98.62:41229 requested a full tablet report, sending...
I20260806 09:02:05.822479 15531 ts_manager.cc:194] Registered new tserver with Master: ba31c2d8ffdb43d7842c643c488bb356 (127.22.98.2:35531)
I20260806 09:02:05.823346 15531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.2:49025
W20260806 09:02:06.175339 15975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:06.175738 15975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:06.175846 15975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:06.183640 15975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:06.183760 15975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:06.183872 15975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:06.193460 15975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:41229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:06.194823 15975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:06.196730 15975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:06.210878 15982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:06.210919 15983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:06.210963 15987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:06.212150 15975 server_base.cc:1034] running on GCE node
I20260806 09:02:06.212836 15975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:06.213927 15975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:06.215186 15975 hybrid_clock.cc:648] HybridClock initialized: now 1786006926215117 us; error 56 us; skew 500 ppm
I20260806 09:02:06.215675 15975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:06.218436 15975 webserver.cc:460] Webserver started at http://127.22.98.3:42607/ using document root <none> and password file <none>
I20260806 09:02:06.219266 15975 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:06.219372 15975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:06.219750 15975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:06.223232 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "5a9ef05e04664762ab75a4ff8caf6974"
format_stamp: "Formatted at 2026-08-06 09:02:06 on dist-test-slave-jpj1"
server_key: "513be60ec3ab2ba7681cb48b1bf7a74d"
server_key_iv: "af93c6d3a4e186de86a90b8ae95ad9e7"
server_key_version: "encryptionkey@0"
I20260806 09:02:06.224293 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "5a9ef05e04664762ab75a4ff8caf6974"
format_stamp: "Formatted at 2026-08-06 09:02:06 on dist-test-slave-jpj1"
server_key: "513be60ec3ab2ba7681cb48b1bf7a74d"
server_key_iv: "af93c6d3a4e186de86a90b8ae95ad9e7"
server_key_version: "encryptionkey@0"
I20260806 09:02:06.230438 15975 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 09:02:06.234560 15996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:06.236238 15975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:06.236428 15975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "5a9ef05e04664762ab75a4ff8caf6974"
format_stamp: "Formatted at 2026-08-06 09:02:06 on dist-test-slave-jpj1"
server_key: "513be60ec3ab2ba7681cb48b1bf7a74d"
server_key_iv: "af93c6d3a4e186de86a90b8ae95ad9e7"
server_key_version: "encryptionkey@0"
I20260806 09:02:06.236665 15975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:06.258493 15975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:06.259475 15975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:06.259826 15975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:06.261022 15975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:06.262753 15975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:06.262852 15975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:06.262970 15975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:06.263046 15975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:06.295359 15975 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:34713
I20260806 09:02:06.295444 16150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:34713 every 8 connection(s)
I20260806 09:02:06.296797 15975 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:06.300987 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 15975
I20260806 09:02:06.301203 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:06.301819 22920 external_mini_cluster.cc:1428] Setting key 7b11cc24e981018d42369ea131dd8d67
I20260806 09:02:06.310521 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.299717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:56015 (local address 127.22.98.62:41229)
0806 09:02:06.299964 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.299969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.301818 (+ 1849us) server_negotiation.cc:407] Connection header received
0806 09:02:06.303880 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.303886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.303971 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.304056 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:06.305433 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.306279 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.307890 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.308369 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.308486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.309870 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:06.309876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:06.309879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:06.309898 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:06.309930 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:06.309942 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:06.310011 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.310188 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.310270 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 09:02:06.311673 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.300236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:41229 (local address 127.22.98.3:56015)
0806 09:02:06.301552 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.301618 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.303528 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.304378 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.304391 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.304520 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:06.305275 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.305296 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.306437 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.306440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.307753 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.307766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.308623 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.308629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.308851 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.309526 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:06.309528 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:06.309570 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:06.309677 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:06.310016 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:06.310022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:06.310027 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.310171 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.310495 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":365,"threads_started":1}
I20260806 09:02:06.313251 16151 heartbeater.cc:347] Connected to a master server at 127.22.98.62:41229
I20260806 09:02:06.313596 16151 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:06.314476 16151 heartbeater.cc:511] Master 127.22.98.62:41229 requested a full tablet report, sending...
I20260806 09:02:06.316394 15531 ts_manager.cc:194] Registered new tserver with Master: 5a9ef05e04664762ab75a4ff8caf6974 (127.22.98.3:34713)
I20260806 09:02:06.317526 15531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.3:56015
I20260806 09:02:06.318317 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:06.330348 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.323451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.22.98.62:41229)
0806 09:02:06.323717 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.323725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.324307 (+ 582us) server_negotiation.cc:407] Connection header received
0806 09:02:06.324478 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.324484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.324587 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.324711 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:06.325283 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.325999 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.327765 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.328263 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.328397 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.329617 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:06.329622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:06.329633 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:06.329656 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:06.329705 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:06.329720 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:06.329802 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.329983 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.330076 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 09:02:06.334758 15531 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:06.342535 15531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:06.346789 15798 heartbeater.cc:503] Master 127.22.98.62:41229 was elected leader, sending a full tablet report...
I20260806 09:02:06.381083 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.356077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.22.98.2:35531)
0806 09:02:06.371357 (+ 15280us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.371366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.371391 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:02:06.371481 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.371487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.371931 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.372133 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:06.373071 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.374630 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.378014 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.379461 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.379676 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.379942 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.380609 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.380805 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15115,"thread_start_us":15475,"threads_started":1}
I20260806 09:02:06.381136 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.356001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53500)
0806 09:02:06.370834 (+ 14833us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.370858 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.371039 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.371939 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.371946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.371979 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:06.372488 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.372503 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.374813 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.374818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.377596 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.377621 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.380023 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.380029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.380273 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.380355 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.380501 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.380725 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14666,"mutex_wait_us":5,"thread_start_us":13328,"threads_started":1}
I20260806 09:02:06.381922 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.355935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:35531 (local address 127.0.0.1:40312)
0806 09:02:06.369812 (+ 13877us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.369867 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.370102 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.372379 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.372383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.372414 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:06.372836 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.372852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.374813 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.374818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.377681 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.377693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.380091 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.380098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.380331 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.380365 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.381392 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.381550 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13538,"mutex_wait_us":680,"thread_start_us":14951,"threads_started":1}
I20260806 09:02:06.382004 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.356704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.22.98.1:45115)
0806 09:02:06.369098 (+ 12394us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.369112 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.371201 (+ 2089us) server_negotiation.cc:407] Connection header received
0806 09:02:06.371321 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.371329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.371653 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.372707 (+ 1054us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:06.372979 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.374644 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.377938 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.379399 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.379564 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.379877 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.381419 (+ 1542us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.381611 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12122,"thread_start_us":13558,"threads_started":1}
I20260806 09:02:06.385860 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.371326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:34713 (local address 127.0.0.1:39776)
0806 09:02:06.377130 (+ 5804us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.377148 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.377293 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.377994 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.377998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.378030 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:06.378426 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.378439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.380380 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.380384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.382418 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.382435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.385010 (+ 2575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.385017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.385257 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.385303 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.385389 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.385530 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5660,"thread_start_us":5505,"threads_started":1}
I20260806 09:02:06.386018 15707 tablet_service.cc:1467] Processing CreateTablet for tablet ba829ed1486f4320b4d64e3e4e26590e (DEFAULT_TABLE table=test-table [id=5793c5a8325a4a56a3de65f3acb61efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:06.386114 15883 tablet_service.cc:1467] Processing CreateTablet for tablet ba829ed1486f4320b4d64e3e4e26590e (DEFAULT_TABLE table=test-table [id=5793c5a8325a4a56a3de65f3acb61efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:06.386888 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.371723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.22.98.3:34713)
0806 09:02:06.375375 (+ 3652us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.375386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.377291 (+ 1905us) server_negotiation.cc:407] Connection header received
0806 09:02:06.377467 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.377474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.377709 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.377913 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:06.378644 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.380189 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.382643 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.384154 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.384299 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.386259 (+ 1960us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.386342 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.386520 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3460,"thread_start_us":3273,"threads_started":1}
I20260806 09:02:06.387509 15707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba829ed1486f4320b4d64e3e4e26590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:06.387506 15883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba829ed1486f4320b4d64e3e4e26590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:06.392163 16066 tablet_service.cc:1467] Processing CreateTablet for tablet ba829ed1486f4320b4d64e3e4e26590e (DEFAULT_TABLE table=test-table [id=5793c5a8325a4a56a3de65f3acb61efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:06.394207 16066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba829ed1486f4320b4d64e3e4e26590e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:06.400045 16182 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Bootstrap starting.
I20260806 09:02:06.404989 16182 tablet_bootstrap.cc:654] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:06.406282 16182 log.cc:824] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:06.409566 16182 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: No bootstrap required, opened a new log
I20260806 09:02:06.409921 16182 ts_tablet_manager.cc:1397] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 09:02:06.410527 16185 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Bootstrap starting.
I20260806 09:02:06.413446 16183 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Bootstrap starting.
I20260806 09:02:06.414094 16185 tablet_bootstrap.cc:654] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:06.415516 16185 log.cc:824] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:06.417338 16183 tablet_bootstrap.cc:654] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:06.418390 16183 log.cc:824] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:06.419267 16185 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: No bootstrap required, opened a new log
I20260806 09:02:06.419633 16185 ts_tablet_manager.cc:1397] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 09:02:06.420100 16182 raft_consensus.cc:348] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.420650 16182 raft_consensus.cc:374] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:06.420805 16182 raft_consensus.cc:729] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8484947393d4f7394be5406bc907422, State: Initialized, Role: FOLLOWER
I20260806 09:02:06.421324 16183 tablet_bootstrap.cc:492] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: No bootstrap required, opened a new log
I20260806 09:02:06.421607 16183 ts_tablet_manager.cc:1397] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260806 09:02:06.421538 16182 consensus_queue.cc:260] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.423847 16182 ts_tablet_manager.cc:1428] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Time spent starting tablet: real 0.014s user 0.004s sys 0.009s
I20260806 09:02:06.430774 16185 raft_consensus.cc:348] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.431447 16185 raft_consensus.cc:374] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:06.431630 16185 raft_consensus.cc:729] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a9ef05e04664762ab75a4ff8caf6974, State: Initialized, Role: FOLLOWER
I20260806 09:02:06.432426 16185 consensus_queue.cc:260] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.433211 16183 raft_consensus.cc:348] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.433744 16183 raft_consensus.cc:374] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:06.433852 16183 raft_consensus.cc:729] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba31c2d8ffdb43d7842c643c488bb356, State: Initialized, Role: FOLLOWER
I20260806 09:02:06.434471 16151 heartbeater.cc:503] Master 127.22.98.62:41229 was elected leader, sending a full tablet report...
I20260806 09:02:06.434677 16183 consensus_queue.cc:260] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.436457 15971 heartbeater.cc:503] Master 127.22.98.62:41229 was elected leader, sending a full tablet report...
I20260806 09:02:06.436957 16183 ts_tablet_manager.cc:1428] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260806 09:02:06.438529 16185 ts_tablet_manager.cc:1428] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Time spent starting tablet: real 0.019s user 0.008s sys 0.006s
I20260806 09:02:06.454027 16189 raft_consensus.cc:482] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:06.454483 16189 raft_consensus.cc:504] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.456677 16189 leader_election.cc:288] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba31c2d8ffdb43d7842c643c488bb356 (127.22.98.2:35531), 5a9ef05e04664762ab75a4ff8caf6974 (127.22.98.3:34713)
I20260806 09:02:06.463073 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.457078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:54865 (local address 127.22.98.2:35531)
0806 09:02:06.457298 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.457305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.457843 (+ 538us) server_negotiation.cc:407] Connection header received
0806 09:02:06.458044 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.458050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.458170 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.458263 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:06.458877 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.459600 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.461413 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.462167 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.462298 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.462400 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.462674 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.462803 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:02:06.463112 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.457044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:35531 (local address 127.22.98.1:54865)
0806 09:02:06.457725 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.457743 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.457902 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.458375 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.458379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.458405 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:06.458732 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.458740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.459740 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.459744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.461217 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.461230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.462363 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.462369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.462510 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.462534 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.462614 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.462762 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":366,"threads_started":1}
I20260806 09:02:06.463529 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.458107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:39175 (local address 127.22.98.3:34713)
0806 09:02:06.458285 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.458291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.458567 (+ 276us) server_negotiation.cc:407] Connection header received
0806 09:02:06.458745 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.458749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.458808 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.458880 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:06.459415 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.460169 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.461876 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.462668 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.462784 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.462883 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.463154 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.463284 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:02:06.463682 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.457845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:34713 (local address 127.22.98.1:39175)
0806 09:02:06.458462 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.458477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:06.458616 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:06.458908 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:06.458912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:06.458935 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:06.459239 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.459249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.460312 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.460317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.461697 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:06.461710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.462861 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.462868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:06.463006 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.463029 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 09:02:06.463117 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 09:02:06.463244 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":283,"threads_started":1}
I20260806 09:02:06.464496 15916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba829ed1486f4320b4d64e3e4e26590e" candidate_uuid: "f8484947393d4f7394be5406bc907422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba31c2d8ffdb43d7842c643c488bb356" is_pre_election: true
I20260806 09:02:06.464893 16092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba829ed1486f4320b4d64e3e4e26590e" candidate_uuid: "f8484947393d4f7394be5406bc907422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a9ef05e04664762ab75a4ff8caf6974" is_pre_election: true
I20260806 09:02:06.465161 15916 raft_consensus.cc:2454] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8484947393d4f7394be5406bc907422 in term 0.
I20260806 09:02:06.465463 16092 raft_consensus.cc:2454] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8484947393d4f7394be5406bc907422 in term 0.
I20260806 09:02:06.465974 15636 leader_election.cc:302] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba31c2d8ffdb43d7842c643c488bb356, f8484947393d4f7394be5406bc907422; no voters:
I20260806 09:02:06.466339 16189 raft_consensus.cc:2781] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:06.466501 16189 raft_consensus.cc:482] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:06.466609 16189 raft_consensus.cc:3037] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:06.468976 16189 raft_consensus.cc:504] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.469619 16189 leader_election.cc:288] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [CANDIDATE]: Term 1 election: Requested vote from peers ba31c2d8ffdb43d7842c643c488bb356 (127.22.98.2:35531), 5a9ef05e04664762ab75a4ff8caf6974 (127.22.98.3:34713)
I20260806 09:02:06.470425 16092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba829ed1486f4320b4d64e3e4e26590e" candidate_uuid: "f8484947393d4f7394be5406bc907422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a9ef05e04664762ab75a4ff8caf6974"
I20260806 09:02:06.470427 15916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba829ed1486f4320b4d64e3e4e26590e" candidate_uuid: "f8484947393d4f7394be5406bc907422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba31c2d8ffdb43d7842c643c488bb356"
I20260806 09:02:06.470682 15916 raft_consensus.cc:3037] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:06.470678 16092 raft_consensus.cc:3037] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:06.474148 15916 raft_consensus.cc:2454] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8484947393d4f7394be5406bc907422 in term 1.
I20260806 09:02:06.474155 16092 raft_consensus.cc:2454] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8484947393d4f7394be5406bc907422 in term 1.
I20260806 09:02:06.474707 15636 leader_election.cc:302] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba31c2d8ffdb43d7842c643c488bb356, f8484947393d4f7394be5406bc907422; no voters:
I20260806 09:02:06.475069 16189 raft_consensus.cc:2781] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:06.475647 16189 raft_consensus.cc:686] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 1 LEADER]: Becoming Leader. State: Replica: f8484947393d4f7394be5406bc907422, State: Running, Role: LEADER
I20260806 09:02:06.476235 16189 consensus_queue.cc:237] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } }
I20260806 09:02:06.481932 15529 catalog_manager.cc:5495] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8484947393d4f7394be5406bc907422 (127.22.98.1). New cstate: current_term: 1 leader_uuid: "f8484947393d4f7394be5406bc907422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8484947393d4f7394be5406bc907422" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 } health_report { overall_health: UNKNOWN } } }
I20260806 09:02:06.514986 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.509298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.22.98.1:45115)
0806 09:02:06.509566 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.509572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.509611 (+ 39us) server_negotiation.cc:407] Connection header received
0806 09:02:06.509887 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.509891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.510012 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.510157 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:06.510585 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.511311 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.512748 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.513183 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.513299 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.513836 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:06.514377 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:06.514522 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.514635 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.514726 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 09:02:06.524300 16092 raft_consensus.cc:1261] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 1 FOLLOWER]: Refusing update from remote peer f8484947393d4f7394be5406bc907422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:06.524299 15916 raft_consensus.cc:1261] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 1 FOLLOWER]: Refusing update from remote peer f8484947393d4f7394be5406bc907422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:06.525314 16197 consensus_queue.cc:1035] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a9ef05e04664762ab75a4ff8caf6974" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:06.525715 16189 consensus_queue.cc:1035] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba31c2d8ffdb43d7842c643c488bb356" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 35531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:06.538096 16205 mvcc.cc:205] Tried to move back new op lower bound from 7315484371032346624 to 7315484370852495360. Current Snapshot: MvccSnapshot[applied={T|T < 7315484371032346624}]
I20260806 09:02:06.540326 16203 mvcc.cc:205] Tried to move back new op lower bound from 7315484371032346624 to 7315484370852495360. Current Snapshot: MvccSnapshot[applied={T|T < 7315484371032346624 or (T in {7315484371032346624})}]
I20260806 09:02:06.553839 15531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:06.554114 15531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:06.557780 15531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fa84f517d1c4bef80ca0ebab65f3e77: Sending DeleteTablet for 3 replicas of tablet ba829ed1486f4320b4d64e3e4e26590e
I20260806 09:02:06.559149 15883 tablet_service.cc:1514] Processing DeleteTablet for tablet ba829ed1486f4320b4d64e3e4e26590e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:06 UTC) from {username='slave'} at 127.0.0.1:40312
I20260806 09:02:06.559157 15707 tablet_service.cc:1514] Processing DeleteTablet for tablet ba829ed1486f4320b4d64e3e4e26590e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:06 UTC) from {username='slave'} at 127.0.0.1:53500
I20260806 09:02:06.559365 16066 tablet_service.cc:1514] Processing DeleteTablet for tablet ba829ed1486f4320b4d64e3e4e26590e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:06 UTC) from {username='slave'} at 127.0.0.1:39776
I20260806 09:02:06.561506 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15616
I20260806 09:02:06.571698 16217 tablet_replica.cc:330] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: stopping tablet replica
I20260806 09:02:06.572295 16217 raft_consensus.cc:2229] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:06.572961 16217 raft_consensus.cc:2258] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:06.581012 16219 tablet_replica.cc:330] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: stopping tablet replica
I20260806 09:02:06.581539 16219 raft_consensus.cc:2229] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:06.582315 16219 raft_consensus.cc:2258] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:06.582985 16217 ts_tablet_manager.cc:1905] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:06.584781 16220 tablet_replica.cc:330] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: stopping tablet replica
I20260806 09:02:06.585287 16220 raft_consensus.cc:2229] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:06.585884 16220 raft_consensus.cc:2258] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:06.586545 16219 ts_tablet_manager.cc:1905] T ba829ed1486f4320b4d64e3e4e26590e P f8484947393d4f7394be5406bc907422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:06.587471 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.563014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.22.98.1:45115)
0806 09:02:06.563221 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.563228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.577128 (+ 13900us) server_negotiation.cc:407] Connection header received
0806 09:02:06.577849 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.577855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.577975 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.578072 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:06.578900 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.580273 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.582593 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.583254 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.583355 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.585452 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:06.585468 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:06.585488 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:06.586300 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:06.586374 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:06.586408 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:06.586497 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.586932 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.587071 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:02:06.588802 16220 ts_tablet_manager.cc:1905] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:06.589011 15789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:06.594134 16217 ts_tablet_manager.cc:1918] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:06.594338 16217 log.cc:1196] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/ba829ed1486f4320b4d64e3e4e26590e
I20260806 09:02:06.594873 16217 ts_tablet_manager.cc:1939] T ba829ed1486f4320b4d64e3e4e26590e P ba31c2d8ffdb43d7842c643c488bb356: Deleting consensus metadata
I20260806 09:02:06.596786 15508 catalog_manager.cc:4841] TS ba31c2d8ffdb43d7842c643c488bb356 (127.22.98.2:35531): tablet ba829ed1486f4320b4d64e3e4e26590e (table test-table [id=5793c5a8325a4a56a3de65f3acb61efc]) successfully deleted
I20260806 09:02:06.597213 16220 ts_tablet_manager.cc:1918] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:06.597460 16220 log.cc:1196] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/ba829ed1486f4320b4d64e3e4e26590e
I20260806 09:02:06.598075 16220 ts_tablet_manager.cc:1939] T ba829ed1486f4320b4d64e3e4e26590e P 5a9ef05e04664762ab75a4ff8caf6974: Deleting consensus metadata
I20260806 09:02:06.600435 15508 catalog_manager.cc:4841] TS 5a9ef05e04664762ab75a4ff8caf6974 (127.22.98.3:34713): tablet ba829ed1486f4320b4d64e3e4e26590e (table test-table [id=5793c5a8325a4a56a3de65f3acb61efc]) successfully deleted
I20260806 09:02:06.690578 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15616
I20260806 09:02:06.714105 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15803
W20260806 09:02:06.714147 15511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:45115 (error 108)
I20260806 09:02:06.716361 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.715815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53548)
0806 09:02:06.716010 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.716158 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 09:02:06.716602 15511 catalog_manager.cc:4584] TS f8484947393d4f7394be5406bc907422 (127.22.98.1:45115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba829ed1486f4320b4d64e3e4e26590e: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
I20260806 09:02:06.721532 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.715056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.22.98.2:35531)
0806 09:02:06.715242 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.715248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.715268 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:02:06.715408 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.715414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.715526 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.715692 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:06.716306 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.717255 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.718618 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.719061 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.719159 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.720165 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:06.720175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:06.720211 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:06.720819 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:06.720876 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:06.720899 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:06.720986 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.721145 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.721262 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:02:06.722262 15965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:06.770538 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.769910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53556)
0806 09:02:06.770227 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.770355 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 09:02:06.771862 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.771401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53558)
0806 09:02:06.771629 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.771718 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 09:02:06.826793 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15803
I20260806 09:02:06.833423 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.832891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53560)
0806 09:02:06.833105 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.833197 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 09:02:06.834492 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.834129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53574)
0806 09:02:06.834307 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.834377 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 09:02:06.850128 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15975
I20260806 09:02:06.856726 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.850853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.22.98.3:34713)
0806 09:02:06.851007 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:06.851012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:06.851031 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:02:06.851187 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:06.851191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:06.851302 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:06.851399 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:06.851784 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.852528 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.853860 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:06.854242 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:06.854334 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:06.855347 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:06.855356 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:06.855373 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:06.856106 (+ 733us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:06.856153 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:06.856175 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:06.856277 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:06.856398 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 09:02:06.856488 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 09:02:06.857393 16144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:06.915355 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.914859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53590)
0806 09:02:06.915084 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.915192 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 09:02:06.916594 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:06.916185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:45115 (local address 127.0.0.1:53600)
0806 09:02:06.916402 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:06.916477 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:45115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 09:02:06.946087 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15975
I20260806 09:02:06.965777 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 15483
I20260806 09:02:06.966912 15580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:07.078437 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 15483
2026-08-06T09:02:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2899 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:07 dist-test-slave-jpj1 krb5kdc[16233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:07 dist-test-slave-jpj1 krb5kdc[16233](info): set up 2 sockets
Aug 06 09:02:07 dist-test-slave-jpj1 krb5kdc[16233](info): commencing operation
krb5kdc: starting...
W20260806 09:02:08.304103 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:08 dist-test-slave-jpj1 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:08.628638 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37597
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:08.986972 16251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:08.987313 16251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:08.987442 16251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:08.995028 16251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:08.995162 16251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:08.995219 16251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:08.995309 16251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:08.995378 16251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:09.005878 16251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:09.007351 16251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:09.008823 16251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:09.021762 16256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:09.021836 16257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:09.021870 16259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:09.022609 16251 server_base.cc:1034] running on GCE node
I20260806 09:02:09.023520 16251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:09.025192 16251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:09.026454 16251 hybrid_clock.cc:648] HybridClock initialized: now 1786006929026374 us; error 68 us; skew 500 ppm
I20260806 09:02:09.026933 16251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:09 dist-test-slave-jpj1 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006929, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:09.034176 16251 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:09.036064 16251 webserver.cc:460] Webserver started at http://127.22.98.62:35285/ using document root <none> and password file <none>
I20260806 09:02:09.036832 16251 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:09.036974 16251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:09.037310 16251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:09.040685 16251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "74381d8f74a74b168d5fec0ea0764134"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "1c0f092e138bd2cdd209502d6aed5e08"
server_key_iv: "a5045da01c947ae6202c9d9c313e1a7d"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.041496 16251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "74381d8f74a74b168d5fec0ea0764134"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "1c0f092e138bd2cdd209502d6aed5e08"
server_key_iv: "a5045da01c947ae6202c9d9c313e1a7d"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.047116 16251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:02:09.050894 16269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:09.052328 16251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:09.052502 16251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "74381d8f74a74b168d5fec0ea0764134"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "1c0f092e138bd2cdd209502d6aed5e08"
server_key_iv: "a5045da01c947ae6202c9d9c313e1a7d"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.052698 16251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:09.068344 16251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:09.073030 16251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:09.073354 16251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:09.092593 16251 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37597
I20260806 09:02:09.092625 16338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37597 every 8 connection(s)
I20260806 09:02:09.094137 16251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:09.098474 16339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:09.100930 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 16251
I20260806 09:02:09.101159 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:09.101724 22920 external_mini_cluster.cc:1428] Setting key 3625230439a1f8e7f8237a0740c77422
I20260806 09:02:09.112731 16339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: Bootstrap starting.
I20260806 09:02:09.117146 16339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: Neither blocks nor log segments found. Creating new log.
Aug 06 09:02:09 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:09.118420 16339 log.cc:824] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:09.122079 16339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: No bootstrap required, opened a new log
I20260806 09:02:09.127566 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:09.103918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52398 (local address 127.22.98.62:37597)
0806 09:02:09.104815 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:09.104828 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:09.104880 (+ 52us) server_negotiation.cc:407] Connection header received
0806 09:02:09.106686 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:09.106714 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:09.107159 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:09.107513 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:09.109512 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.110993 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.113078 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.113873 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.114109 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:09.119418 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:09.119444 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:09.119467 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:09.119526 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:09.122864 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:09.123917 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:09.123925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:09.123936 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:09.124131 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:09.124564 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:09.124567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:09.124569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:09.125176 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:09.125451 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:09.125948 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 09:02:09.126376 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":413,"threads_started":1}
I20260806 09:02:09.131836 16339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } }
I20260806 09:02:09.132269 16339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:09.132376 16339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74381d8f74a74b168d5fec0ea0764134, State: Initialized, Role: FOLLOWER
I20260806 09:02:09.133009 16339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } }
I20260806 09:02:09.133268 16339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:09.133402 16339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:09.133564 16339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:09.135967 16339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } }
I20260806 09:02:09.136497 16339 leader_election.cc:302] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74381d8f74a74b168d5fec0ea0764134; no voters:
I20260806 09:02:09.137241 16339 leader_election.cc:288] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:09.137411 16347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:09.138666 16347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [term 1 LEADER]: Becoming Leader. State: Replica: 74381d8f74a74b168d5fec0ea0764134, State: Running, Role: LEADER
I20260806 09:02:09.139240 16347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } }
I20260806 09:02:09.139686 16339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:09.142310 16350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74381d8f74a74b168d5fec0ea0764134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } } }
I20260806 09:02:09.142321 16351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74381d8f74a74b168d5fec0ea0764134. Latest consensus state: current_term: 1 leader_uuid: "74381d8f74a74b168d5fec0ea0764134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74381d8f74a74b168d5fec0ea0764134" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37597 } } }
I20260806 09:02:09.142863 16350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:09.142949 16351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:09.143778 16355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:09.149818 16355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:09.162968 16355 catalog_manager.cc:1347] Generated new cluster ID: 198b4de2c824457281d7e56fe58c58a2
I20260806 09:02:09.163079 16355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:09.177563 16355 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:09.179150 16355 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:09.189168 16355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: Generated new TSK 0
I20260806 09:02:09.190055 16355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:02:09.262692 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:09.640635 16378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:09.640955 16378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:09.641069 16378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:09.648773 16378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:09.648903 16378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:09.649026 16378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:09.659224 16378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:09.660640 16378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:09.662031 16378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:09.676319 16386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:09.676362 16387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:09.676326 16391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:09.677903 16378 server_base.cc:1034] running on GCE node
I20260806 09:02:09.678640 16378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:09.679747 16378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:09.680985 16378 hybrid_clock.cc:648] HybridClock initialized: now 1786006929680912 us; error 67 us; skew 500 ppm
I20260806 09:02:09.681356 16378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:09 dist-test-slave-jpj1 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006929, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:09.684907 16378 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:02:09.686631 16378 webserver.cc:460] Webserver started at http://127.22.98.1:35665/ using document root <none> and password file <none>
I20260806 09:02:09.687383 16378 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:09.687502 16378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:09.687857 16378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:09.691301 16378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "584ca6f9f330480cb43c11139a6c9407"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "3efc58a60e1b600a11f12b9d9dd7a882"
server_key_iv: "ff63118d3c6b1162190ab5b3b7acfaf3"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.692185 16378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "584ca6f9f330480cb43c11139a6c9407"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "3efc58a60e1b600a11f12b9d9dd7a882"
server_key_iv: "ff63118d3c6b1162190ab5b3b7acfaf3"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.698056 16378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:02:09.702193 16400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:09.703828 16378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:02:09.704003 16378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "584ca6f9f330480cb43c11139a6c9407"
format_stamp: "Formatted at 2026-08-06 09:02:09 on dist-test-slave-jpj1"
server_key: "3efc58a60e1b600a11f12b9d9dd7a882"
server_key_iv: "ff63118d3c6b1162190ab5b3b7acfaf3"
server_key_version: "encryptionkey@0"
I20260806 09:02:09.704193 16378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:09.737735 16378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:09.743026 16378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:09.743316 16378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:09.744454 16378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:09.745926 16378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:09.746006 16378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:09.746102 16378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:09.746147 16378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:09.780369 16378 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:36093
I20260806 09:02:09.780448 16549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:36093 every 8 connection(s)
I20260806 09:02:09.781734 16378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:09.791785 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 16378
I20260806 09:02:09.791965 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:09.792452 22920 external_mini_cluster.cc:1428] Setting key 14d6728c24314a203bdb01b7b7fd82a8
Aug 06 09:02:09 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006929, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:09.802901 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:09.784456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:51633 (local address 127.22.98.62:37597)
0806 09:02:09.784969 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:09.784974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:09.786016 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 09:02:09.788005 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:09.788011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:09.788102 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:09.788205 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:09.789912 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.790671 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.792464 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.792917 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.793046 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:09.797027 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:09.797033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:09.797036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:09.797072 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:09.799645 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:09.800464 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:09.800469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:09.800471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:09.800601 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:09.801055 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:09.801059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:09.801062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:09.801358 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:09.801509 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:09.802295 (+ 786us) server_negotiation.cc:299] Negotiation successful
0806 09:02:09.802537 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":274,"threads_started":1}
I20260806 09:02:09.804018 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:09.785037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37597 (local address 127.22.98.1:51633)
0806 09:02:09.785827 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:09.785876 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:09.787695 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:09.788480 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:09.788490 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:09.789071 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:09.789703 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:09.789723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.790861 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.790866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:09.792305 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:09.792320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:09.793182 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:09.793191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:09.793442 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:09.794176 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:09.794208 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:09.796759 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:09.799814 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:09.799825 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:09.799843 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:09.800320 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:09.800716 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:09.800722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:09.800726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:09.800930 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:09.801494 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:09.801503 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:09.801886 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 09:02:09.802148 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 09:02:09.802742 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":309,"threads_started":1}
I20260806 09:02:09.805648 16550 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37597
I20260806 09:02:09.805994 16550 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:09.806886 16550 heartbeater.cc:511] Master 127.22.98.62:37597 requested a full tablet report, sending...
I20260806 09:02:09.809518 16291 ts_manager.cc:194] Registered new tserver with Master: 584ca6f9f330480cb43c11139a6c9407 (127.22.98.1:36093)
I20260806 09:02:09.811596 16291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:51633
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 09:02:09.863119 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:10.222178 16558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:10.222476 16558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:10.222568 16558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:10.229969 16558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:10.230103 16558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:10.230219 16558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:10.240278 16558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:10.241695 16558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:10.243013 16558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:10.256548 16567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:10.256536 16569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:10.256548 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:10.257685 16558 server_base.cc:1034] running on GCE node
I20260806 09:02:10.258399 16558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:10.259434 16558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:10.260654 16558 hybrid_clock.cc:648] HybridClock initialized: now 1786006930260586 us; error 62 us; skew 500 ppm
I20260806 09:02:10.261006 16558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:10 dist-test-slave-jpj1 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006930, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:10.264259 16558 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:02:10.265825 16558 webserver.cc:460] Webserver started at http://127.22.98.2:43367/ using document root <none> and password file <none>
I20260806 09:02:10.266521 16558 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:10.266628 16558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:10.266920 16558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:10.269948 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "ca56d8a32fb7437886c5c5d52b9b69ab"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "4e23fd41542cba7503689fc51278ce78"
server_key_iv: "e1fc5117f154774834100c43dfe7be59"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.270684 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "ca56d8a32fb7437886c5c5d52b9b69ab"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "4e23fd41542cba7503689fc51278ce78"
server_key_iv: "e1fc5117f154774834100c43dfe7be59"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.276082 16558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:02:10.279595 16579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.281039 16558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:10.281196 16558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "ca56d8a32fb7437886c5c5d52b9b69ab"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "4e23fd41542cba7503689fc51278ce78"
server_key_iv: "e1fc5117f154774834100c43dfe7be59"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.281394 16558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:10.295284 16558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:10.299822 16558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:10.300107 16558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:10.301134 16558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:10.302565 16558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:10.302644 16558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.302748 16558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:10.302810 16558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.334062 16558 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39725
I20260806 09:02:10.334107 16729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39725 every 8 connection(s)
I20260806 09:02:10.335345 16558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:10.345752 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 16558
I20260806 09:02:10.345944 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:10.346386 22920 external_mini_cluster.cc:1428] Setting key 6409d76b7e06905f2942b5ef3852e452
Aug 06 09:02:10 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006930, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:10.355720 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.337926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:37439 (local address 127.22.98.62:37597)
0806 09:02:10.338366 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.338371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.339253 (+ 882us) server_negotiation.cc:407] Connection header received
0806 09:02:10.341201 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.341208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.341293 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.341394 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:10.342917 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.343686 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.345225 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.345680 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.345806 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.349858 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:10.349864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:10.349867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:10.349901 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:10.352530 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.353375 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.353379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.353381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.353511 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.353979 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.353982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.353984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.354248 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:10.354364 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.355194 (+ 830us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.355397 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":230,"threads_started":1}
I20260806 09:02:10.357185 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.338355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37597 (local address 127.22.98.2:37439)
0806 09:02:10.339084 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:10.339124 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:10.340920 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:10.341639 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:10.341647 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:10.342159 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:10.342736 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.342755 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.343844 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.343848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.345075 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.345089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.345914 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.345922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.346178 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.346820 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:10.346845 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:10.349544 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:10.352683 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:10.352692 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:10.352709 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:10.353268 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:10.353612 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:10.353617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:10.353620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:10.353878 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:10.354371 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:10.354378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:10.354822 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 09:02:10.355130 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 09:02:10.355721 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":278,"threads_started":1}
I20260806 09:02:10.358919 16731 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37597
I20260806 09:02:10.359337 16731 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:10.360388 16731 heartbeater.cc:511] Master 127.22.98.62:37597 requested a full tablet report, sending...
I20260806 09:02:10.362272 16291 ts_manager.cc:194] Registered new tserver with Master: ca56d8a32fb7437886c5c5d52b9b69ab (127.22.98.2:39725)
I20260806 09:02:10.363255 16291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:37439
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:02:10.414506 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:10.765996 16740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:10.766333 16740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:10.766481 16740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:10.774103 16740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:10.774257 16740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:10.774420 16740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:10.785059 16740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:10.786482 16740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:10.787987 16740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:10.801980 16747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:10.803958 16750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:10.803992 16748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:10.805361 16740 server_base.cc:1034] running on GCE node
I20260806 09:02:10.806097 16740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:10.807152 16740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:10.808378 16740 hybrid_clock.cc:648] HybridClock initialized: now 1786006930808307 us; error 70 us; skew 500 ppm
I20260806 09:02:10.808856 16740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:10 dist-test-slave-jpj1 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006930, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:10.812373 16740 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:02:10.813982 16740 webserver.cc:460] Webserver started at http://127.22.98.3:35799/ using document root <none> and password file <none>
I20260806 09:02:10.814713 16740 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:10.814826 16740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:10.815122 16740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:10.815682 16550 heartbeater.cc:503] Master 127.22.98.62:37597 was elected leader, sending a full tablet report...
I20260806 09:02:10.818864 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "0d8975249175402c9d4523457710dd62"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "8ed40c5866c4cbbe80be7849dfe9e357"
server_key_iv: "879aee3d3008059c638862ebe16f8801"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.819759 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "0d8975249175402c9d4523457710dd62"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "8ed40c5866c4cbbe80be7849dfe9e357"
server_key_iv: "879aee3d3008059c638862ebe16f8801"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.825533 16740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:02:10.829703 16761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.831336 16740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:02:10.831523 16740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "0d8975249175402c9d4523457710dd62"
format_stamp: "Formatted at 2026-08-06 09:02:10 on dist-test-slave-jpj1"
server_key: "8ed40c5866c4cbbe80be7849dfe9e357"
server_key_iv: "879aee3d3008059c638862ebe16f8801"
server_key_version: "encryptionkey@0"
I20260806 09:02:10.831770 16740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:10.848193 16740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:10.853032 16740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:10.853341 16740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:10.854455 16740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:10.856062 16740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:10.856153 16740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.856276 16740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:10.856333 16740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:10.889846 16740 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:45887
I20260806 09:02:10.889911 16914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:45887 every 8 connection(s)
I20260806 09:02:10.891311 16740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:10.896440 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 16740
I20260806 09:02:10.896612 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:10.897068 22920 external_mini_cluster.cc:1428] Setting key a4fe26724ceee194aa945263f5c3c97d
Aug 06 09:02:10 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006930, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:10.912573 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.894377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:49181 (local address 127.22.98.62:37597)
0806 09:02:10.894922 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.894928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.895982 (+ 1054us) server_negotiation.cc:407] Connection header received
0806 09:02:10.898123 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.898129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.898244 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.898344 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:10.900092 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.900877 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.902484 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.902895 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.903007 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.906527 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:10.906532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:10.906536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:10.906572 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:10.909295 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.910092 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.910097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.910099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.910233 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.910685 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.910689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.910692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.910972 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:10.911128 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.911941 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.912184 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":288,"threads_started":1}
I20260806 09:02:10.913578 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.894850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37597 (local address 127.22.98.3:49181)
0806 09:02:10.895789 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:10.895838 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:10.897822 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:10.898660 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:10.898670 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:10.899233 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:10.899917 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.899937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.901045 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.901049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.902330 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.902345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.903130 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.903137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.903376 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.904079 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:10.904104 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:10.906296 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:10.909443 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:10.909454 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:10.909474 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:10.909959 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:10.910343 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:10.910348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:10.910353 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:10.910564 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:10.911104 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:10.911113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:10.911477 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 09:02:10.911811 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 09:02:10.912366 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":335,"threads_started":1}
I20260806 09:02:10.915156 16916 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37597
I20260806 09:02:10.915515 16916 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:10.916376 16916 heartbeater.cc:511] Master 127.22.98.62:37597 requested a full tablet report, sending...
I20260806 09:02:10.918030 16291 ts_manager.cc:194] Registered new tserver with Master: 0d8975249175402c9d4523457710dd62 (127.22.98.3:45887)
I20260806 09:02:10.918859 16291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:49181
I20260806 09:02:10.925290 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:10.943070 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.929656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53994 (local address 127.22.98.62:37597)
0806 09:02:10.929835 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.929840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.930207 (+ 367us) server_negotiation.cc:407] Connection header received
0806 09:02:10.930409 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.930414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.930510 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.930604 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:10.932054 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.932762 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.934110 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.934559 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.934687 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.937620 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:10.937625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:10.937629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:10.937665 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:10.940373 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.941301 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.941305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.941308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.941441 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:10.941814 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:10.941817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:10.941819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:10.942128 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:10.942356 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.942680 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.942830 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:02:10.947172 16291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:10.954370 16291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:10.985814 16936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.967501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:45887 (local address 127.0.0.1:35142)
0806 09:02:10.970074 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:10.970126 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:10.970326 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:10.973725 (+ 3399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:10.973735 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:10.973783 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:10.974283 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.974297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.978296 (+ 3999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.978303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.982574 (+ 4271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.982587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.984520 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.984527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.984740 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.984813 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 09:02:10.984948 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 09:02:10.985182 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2260,"mutex_wait_us":1,"thread_start_us":13961,"threads_started":1}
I20260806 09:02:10.986480 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.967732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.22.98.3:45887)
0806 09:02:10.972638 (+ 4906us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.972649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.972693 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:02:10.972812 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.972821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.973126 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.973350 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:10.974533 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.978043 (+ 3510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.982764 (+ 4721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.984266 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.984398 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.984649 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.985952 (+ 1303us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.986146 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4692,"thread_start_us":3037,"threads_started":1}
I20260806 09:02:10.989897 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.971360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.22.98.1:36093)
0806 09:02:10.981588 (+ 10228us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.981599 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.982162 (+ 563us) server_negotiation.cc:407] Connection header received
0806 09:02:10.982323 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.982329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.982523 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.982735 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:10.983434 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.984660 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.987577 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.988823 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.988948 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.989189 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.989392 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.989568 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9995,"thread_start_us":10233,"threads_started":1}
I20260806 09:02:10.989925 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.970589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55156)
0806 09:02:10.982006 (+ 11417us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:10.982024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:10.982163 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:10.982757 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:10.982761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:10.982789 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:10.983220 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.983237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.984799 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.984805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.987367 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.987384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.989028 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.989035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.989208 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.989244 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:02:10.989333 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 09:02:10.989472 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11288,"mutex_wait_us":27,"thread_start_us":10445,"threads_started":1}
I20260806 09:02:10.990830 16934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.967629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39725 (local address 127.0.0.1:35972)
0806 09:02:10.981813 (+ 14184us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:10.981835 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:10.981996 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:10.983000 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:10.983003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:10.983029 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:10.983414 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.983425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.985262 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.985267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.988031 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:10.988043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.990064 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.990071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:10.990245 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.990279 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:02:10.990368 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 09:02:10.990495 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14047,"thread_start_us":3972,"threads_started":1}
I20260806 09:02:10.991688 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:10.968117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.22.98.2:39725)
0806 09:02:10.969408 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:10.969420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:10.981982 (+ 12562us) server_negotiation.cc:407] Connection header received
0806 09:02:10.982181 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:10.982191 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:10.982512 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:10.982735 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:10.983615 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.985080 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.988192 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:10.989598 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:10.989730 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:10.989912 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:10.991149 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0806 09:02:10.991337 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":886,"threads_started":1}
I20260806 09:02:10.991900 16828 tablet_service.cc:1467] Processing CreateTablet for tablet a21a4538526e47d3b917238d15364a12 (DEFAULT_TABLE table=test-table [id=83add988835c43b38a6e76c765183706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:10.993395 16828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21a4538526e47d3b917238d15364a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:10.993700 16465 tablet_service.cc:1467] Processing CreateTablet for tablet a21a4538526e47d3b917238d15364a12 (DEFAULT_TABLE table=test-table [id=83add988835c43b38a6e76c765183706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:10.994884 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21a4538526e47d3b917238d15364a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:10.999297 16641 tablet_service.cc:1467] Processing CreateTablet for tablet a21a4538526e47d3b917238d15364a12 (DEFAULT_TABLE table=test-table [id=83add988835c43b38a6e76c765183706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:11.000906 16641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21a4538526e47d3b917238d15364a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:11.005339 16942 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Bootstrap starting.
I20260806 09:02:11.009195 16942 tablet_bootstrap.cc:654] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:11.010454 16942 log.cc:824] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:11.013458 16942 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: No bootstrap required, opened a new log
I20260806 09:02:11.013723 16942 ts_tablet_manager.cc:1397] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 09:02:11.015393 16947 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Bootstrap starting.
I20260806 09:02:11.015690 16943 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Bootstrap starting.
I20260806 09:02:11.018693 16947 tablet_bootstrap.cc:654] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:11.019452 16943 tablet_bootstrap.cc:654] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:11.020434 16947 log.cc:824] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:11.020761 16943 log.cc:824] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:11.023639 16943 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: No bootstrap required, opened a new log
I20260806 09:02:11.023895 16947 tablet_bootstrap.cc:492] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: No bootstrap required, opened a new log
I20260806 09:02:11.023924 16943 ts_tablet_manager.cc:1397] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 09:02:11.024194 16947 ts_tablet_manager.cc:1397] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 09:02:11.023892 16942 raft_consensus.cc:348] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.024400 16942 raft_consensus.cc:374] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:11.024515 16942 raft_consensus.cc:729] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8975249175402c9d4523457710dd62, State: Initialized, Role: FOLLOWER
I20260806 09:02:11.025245 16942 consensus_queue.cc:260] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.026505 16916 heartbeater.cc:503] Master 127.22.98.62:37597 was elected leader, sending a full tablet report...
I20260806 09:02:11.027244 16942 ts_tablet_manager.cc:1428] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 09:02:11.033435 16947 raft_consensus.cc:348] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.033998 16947 raft_consensus.cc:374] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:11.034106 16947 raft_consensus.cc:729] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca56d8a32fb7437886c5c5d52b9b69ab, State: Initialized, Role: FOLLOWER
I20260806 09:02:11.033819 16943 raft_consensus.cc:348] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.034406 16943 raft_consensus.cc:374] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:11.034511 16943 raft_consensus.cc:729] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584ca6f9f330480cb43c11139a6c9407, State: Initialized, Role: FOLLOWER
I20260806 09:02:11.034875 16947 consensus_queue.cc:260] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.035015 16943 consensus_queue.cc:260] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.036938 16943 ts_tablet_manager.cc:1428] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 09:02:11.037923 16731 heartbeater.cc:503] Master 127.22.98.62:37597 was elected leader, sending a full tablet report...
I20260806 09:02:11.038424 16947 ts_tablet_manager.cc:1428] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260806 09:02:11.087468 16732 tablet.cc:2369] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:11.143326 16917 tablet.cc:2369] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:11.249723 16952 raft_consensus.cc:482] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:11.250058 16952 raft_consensus.cc:504] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.252352 16952 leader_election.cc:288] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca56d8a32fb7437886c5c5d52b9b69ab (127.22.98.2:39725), 584ca6f9f330480cb43c11139a6c9407 (127.22.98.1:36093)
I20260806 09:02:11.259372 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.253015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.22.98.3:36387)
0806 09:02:11.253213 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.253230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:11.253365 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:11.253798 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:11.253804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:11.253837 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:11.254072 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:11.254085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.255358 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.255362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:11.257265 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:11.257284 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.258752 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.258758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:11.258907 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.258935 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 09:02:11.259023 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 09:02:11.259125 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":6}
I20260806 09:02:11.259464 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.252873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:36387 (local address 127.22.98.1:36093)
0806 09:02:11.253102 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.253108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.253323 (+ 215us) server_negotiation.cc:407] Connection header received
0806 09:02:11.253536 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.253542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.253634 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.253747 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:11.254247 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.255198 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.257471 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.258521 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.258684 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.258797 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.259073 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.259212 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:02:11.260277 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.252916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:50839 (local address 127.22.98.2:39725)
0806 09:02:11.253119 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.253125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.254248 (+ 1123us) server_negotiation.cc:407] Connection header received
0806 09:02:11.254393 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.254398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.254478 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.254579 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:11.255121 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.255923 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.258170 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.259283 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.259421 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.259517 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.259884 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.260000 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:02:11.260347 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.253390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39725 (local address 127.22.98.3:50839)
0806 09:02:11.254127 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.254153 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:11.254271 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:11.254651 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:11.254654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:11.254678 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:11.254968 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:11.254976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.256136 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.256142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:11.257973 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:11.257987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.259515 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.259523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:11.259713 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.259748 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:02:11.259842 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 09:02:11.259970 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":45,"thread_start_us":406,"threads_started":1}
I20260806 09:02:11.261050 16490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21a4538526e47d3b917238d15364a12" candidate_uuid: "0d8975249175402c9d4523457710dd62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584ca6f9f330480cb43c11139a6c9407" is_pre_election: true
I20260806 09:02:11.261335 16668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21a4538526e47d3b917238d15364a12" candidate_uuid: "0d8975249175402c9d4523457710dd62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" is_pre_election: true
I20260806 09:02:11.261585 16490 raft_consensus.cc:2454] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d8975249175402c9d4523457710dd62 in term 0.
I20260806 09:02:11.261873 16668 raft_consensus.cc:2454] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d8975249175402c9d4523457710dd62 in term 0.
I20260806 09:02:11.262456 16764 leader_election.cc:302] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d8975249175402c9d4523457710dd62, 584ca6f9f330480cb43c11139a6c9407; no voters:
I20260806 09:02:11.262902 16952 raft_consensus.cc:2781] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:11.263043 16952 raft_consensus.cc:482] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:11.263141 16952 raft_consensus.cc:3037] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:11.265403 16952 raft_consensus.cc:504] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.266100 16952 leader_election.cc:288] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [CANDIDATE]: Term 1 election: Requested vote from peers ca56d8a32fb7437886c5c5d52b9b69ab (127.22.98.2:39725), 584ca6f9f330480cb43c11139a6c9407 (127.22.98.1:36093)
I20260806 09:02:11.266615 16668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21a4538526e47d3b917238d15364a12" candidate_uuid: "0d8975249175402c9d4523457710dd62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab"
I20260806 09:02:11.266768 16490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21a4538526e47d3b917238d15364a12" candidate_uuid: "0d8975249175402c9d4523457710dd62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584ca6f9f330480cb43c11139a6c9407"
I20260806 09:02:11.266913 16668 raft_consensus.cc:3037] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:11.266996 16490 raft_consensus.cc:3037] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:11.269342 16668 raft_consensus.cc:2454] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d8975249175402c9d4523457710dd62 in term 1.
I20260806 09:02:11.269376 16490 raft_consensus.cc:2454] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d8975249175402c9d4523457710dd62 in term 1.
I20260806 09:02:11.269784 16764 leader_election.cc:302] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d8975249175402c9d4523457710dd62, 584ca6f9f330480cb43c11139a6c9407; no voters:
I20260806 09:02:11.270088 16952 raft_consensus.cc:2781] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:11.270623 16952 raft_consensus.cc:686] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 1 LEADER]: Becoming Leader. State: Replica: 0d8975249175402c9d4523457710dd62, State: Running, Role: LEADER
I20260806 09:02:11.271118 16952 consensus_queue.cc:237] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } }
I20260806 09:02:11.275733 16291 catalog_manager.cc:5495] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d8975249175402c9d4523457710dd62 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "0d8975249175402c9d4523457710dd62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d8975249175402c9d4523457710dd62" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 45887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 } health_report { overall_health: UNKNOWN } } }
W20260806 09:02:11.284425 16551 tablet.cc:2369] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:11.352761 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.346877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.22.98.3:45887)
0806 09:02:11.347052 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.347056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.347140 (+ 84us) server_negotiation.cc:407] Connection header received
0806 09:02:11.347321 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.347325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.347408 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.347588 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:11.347948 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.348718 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.350172 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.350591 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.350734 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.351484 (+ 750us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:11.352107 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:11.352248 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.352487 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.352535 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:02:11.362110 16490 raft_consensus.cc:1261] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 1 FOLLOWER]: Refusing update from remote peer 0d8975249175402c9d4523457710dd62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:11.362116 16668 raft_consensus.cc:1261] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 1 FOLLOWER]: Refusing update from remote peer 0d8975249175402c9d4523457710dd62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:11.363436 16956 consensus_queue.cc:1035] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca56d8a32fb7437886c5c5d52b9b69ab" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:11.363909 16952 consensus_queue.cc:1035] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "584ca6f9f330480cb43c11139a6c9407" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:11.378055 16960 mvcc.cc:205] Tried to move back new op lower bound from 7315484390846414848 to 7315484390490419200. Current Snapshot: MvccSnapshot[applied={T|T < 7315484390846414848}]
I20260806 09:02:11.390914 16290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:11.391269 16290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:11.395673 16290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74381d8f74a74b168d5fec0ea0764134: Sending DeleteTablet for 3 replicas of tablet a21a4538526e47d3b917238d15364a12
I20260806 09:02:11.396922 16828 tablet_service.cc:1514] Processing DeleteTablet for tablet a21a4538526e47d3b917238d15364a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35142
I20260806 09:02:11.397007 16465 tablet_service.cc:1514] Processing DeleteTablet for tablet a21a4538526e47d3b917238d15364a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:55156
I20260806 09:02:11.397710 16641 tablet_service.cc:1514] Processing DeleteTablet for tablet a21a4538526e47d3b917238d15364a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:11 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:35972
I20260806 09:02:11.399116 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 16378
I20260806 09:02:11.418875 16975 tablet_replica.cc:330] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: stopping tablet replica
I20260806 09:02:11.419482 16975 raft_consensus.cc:2229] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:11.420161 16975 raft_consensus.cc:2258] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:11.421027 16974 tablet_replica.cc:330] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: stopping tablet replica
I20260806 09:02:11.421532 16974 raft_consensus.cc:2229] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:11.422165 16978 tablet_replica.cc:330] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: stopping tablet replica
I20260806 09:02:11.422351 16974 raft_consensus.cc:2258] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:11.422585 16978 raft_consensus.cc:2229] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:11.423131 16978 raft_consensus.cc:2258] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:11.425333 16978 ts_tablet_manager.cc:1905] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 09:02:11 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:02:11.426334 16974 ts_tablet_manager.cc:1905] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:11.432642 16978 ts_tablet_manager.cc:1918] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:11.432842 16978 log.cc:1196] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/a21a4538526e47d3b917238d15364a12
I20260806 09:02:11.433451 16978 ts_tablet_manager.cc:1939] T a21a4538526e47d3b917238d15364a12 P ca56d8a32fb7437886c5c5d52b9b69ab: Deleting consensus metadata
I20260806 09:02:11.434059 16974 ts_tablet_manager.cc:1918] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:11.434273 16974 log.cc:1196] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/a21a4538526e47d3b917238d15364a12
I20260806 09:02:11.434894 16974 ts_tablet_manager.cc:1939] T a21a4538526e47d3b917238d15364a12 P 0d8975249175402c9d4523457710dd62: Deleting consensus metadata
I20260806 09:02:11.435684 16271 catalog_manager.cc:4841] TS ca56d8a32fb7437886c5c5d52b9b69ab (127.22.98.2:39725): tablet a21a4538526e47d3b917238d15364a12 (table test-table [id=83add988835c43b38a6e76c765183706]) successfully deleted
I20260806 09:02:11.436223 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.400350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55164 (local address 127.22.98.1:36093)
0806 09:02:11.400545 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.400551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.412019 (+ 11468us) server_negotiation.cc:407] Connection header received
0806 09:02:11.412615 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.412623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.412761 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.412853 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:11.414957 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.416359 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.419296 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.420030 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.420151 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.426927 (+ 6776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:11.426946 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:11.426964 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:11.427023 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:11.431425 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.432541 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.432552 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.432556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.432895 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.433472 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.433478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.433482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.434144 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:11.434630 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.435414 (+ 784us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.435758 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:02:11.436417 16975 ts_tablet_manager.cc:1905] T a21a4538526e47d3b917238d15364a12 P 584ca6f9f330480cb43c11139a6c9407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:11.437635 16545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:11.438462 16272 catalog_manager.cc:4841] TS 0d8975249175402c9d4523457710dd62 (127.22.98.3:45887): tablet a21a4538526e47d3b917238d15364a12 (table test-table [id=83add988835c43b38a6e76c765183706]) successfully deleted
I20260806 09:02:11.579159 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 16378
I20260806 09:02:11.602823 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 16558
W20260806 09:02:11.602962 16273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:36093 (error 108)
I20260806 09:02:11.605259 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.604334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55176)
0806 09:02:11.604877 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.605015 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":318,"threads_started":1}
W20260806 09:02:11.605573 16273 catalog_manager.cc:4584] TS 584ca6f9f330480cb43c11139a6c9407 (127.22.98.1:36093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a21a4538526e47d3b917238d15364a12: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Aug 06 09:02:11 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:02:11.621110 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.603653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.22.98.2:39725)
0806 09:02:11.603861 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.603868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.604145 (+ 277us) server_negotiation.cc:407] Connection header received
0806 09:02:11.604231 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.604236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.604306 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.604408 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:11.606213 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.606932 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.608464 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.608869 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.608988 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.613690 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:11.613713 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:11.613737 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:11.613790 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:11.617689 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.618533 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.618544 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.618548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.618742 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.619148 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.619153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.619157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.619706 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:11.620028 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.620475 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.620721 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:02:11.621841 16721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:11.659471 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.658911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55182)
0806 09:02:11.659165 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.659294 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 09:02:11.660796 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.660334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55186)
0806 09:02:11.660505 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.660604 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 09:02:11.712922 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 16558
I20260806 09:02:11.725214 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.724584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55198)
0806 09:02:11.724873 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.725002 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 09:02:11.727523 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.727058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55204)
0806 09:02:11.727240 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.727338 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:02:11.744151 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 16740
Aug 06 09:02:11 dist-test-slave-jpj1 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:02:11.764081 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.744912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.22.98.3:45887)
0806 09:02:11.745130 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:11.745136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:11.745157 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:02:11.745271 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:11.745277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:11.745364 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:11.745462 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:11.746806 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.747913 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.750011 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:11.750498 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:11.750611 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:11.756624 (+ 6013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:11.756642 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:11.756684 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:11.756742 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:11.760488 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.761551 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.761560 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.761563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.761733 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:11.762309 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:11.762316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:11.762320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:11.762768 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:11.763123 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:11.763425 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 09:02:11.763731 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 09:02:11.764899 16910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:11.808609 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.808096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55212)
0806 09:02:11.808360 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.808454 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 09:02:11.809764 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:11.809290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55222)
0806 09:02:11.809513 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:11.809600 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 09:02:11.891090 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 16740
I20260806 09:02:11.915098 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 16251
I20260806 09:02:11.916352 16334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:12.029230 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:12.028606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:36093 (local address 127.0.0.1:55238)
0806 09:02:12.028933 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:12.029043 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:36093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 09:02:12.029589 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 16251
2026-08-06T09:02:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4957 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T09:02:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:12Z Disabled control of system clock
I20260806 09:02:12.133272 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37713
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:12.508415 16992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:12.508778 16992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:12.508881 16992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:12.516575 16992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:12.516712 16992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:12.516750 16992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:12.516819 16992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:12.516865 16992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:12.527426 16992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:12.530395 16992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:12.532568 16992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:12.544970 17000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:12.545102 17004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:12.545131 17001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:12.546394 16992 server_base.cc:1034] running on GCE node
I20260806 09:02:12.547240 16992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:12.548914 16992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:12.550173 16992 hybrid_clock.cc:648] HybridClock initialized: now 1786006932550102 us; error 62 us; skew 500 ppm
I20260806 09:02:12.550571 16992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:12.553269 16992 webserver.cc:460] Webserver started at http://127.22.98.62:34075/ using document root <none> and password file <none>
I20260806 09:02:12.554106 16992 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:12.554240 16992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:12.554685 16992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:12.558319 16992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "a0d938a5f99e4b508e5df91ebb45df51"
format_stamp: "Formatted at 2026-08-06 09:02:12 on dist-test-slave-jpj1"
server_key: "24cb4ba8e854c5495cfa901ab337a331"
server_key_iv: "d3748455e6756d5c8f010b14eef9eea8"
server_key_version: "encryptionkey@0"
I20260806 09:02:12.559180 16992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "a0d938a5f99e4b508e5df91ebb45df51"
format_stamp: "Formatted at 2026-08-06 09:02:12 on dist-test-slave-jpj1"
server_key: "24cb4ba8e854c5495cfa901ab337a331"
server_key_iv: "d3748455e6756d5c8f010b14eef9eea8"
server_key_version: "encryptionkey@0"
I20260806 09:02:12.565228 16992 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 09:02:12.569208 17012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:12.570870 16992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:12.571061 16992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "a0d938a5f99e4b508e5df91ebb45df51"
format_stamp: "Formatted at 2026-08-06 09:02:12 on dist-test-slave-jpj1"
server_key: "24cb4ba8e854c5495cfa901ab337a331"
server_key_iv: "d3748455e6756d5c8f010b14eef9eea8"
server_key_version: "encryptionkey@0"
I20260806 09:02:12.571301 16992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:12.586381 16992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:12.587266 16992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:12.587581 16992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:12.606833 16992 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37713
I20260806 09:02:12.606885 17091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37713 every 8 connection(s)
I20260806 09:02:12.608314 16992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:12.612459 17092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:12.616722 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 16992
I20260806 09:02:12.616928 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:12.617496 22920 external_mini_cluster.cc:1428] Setting key 0ee16182c27eef6376d0ba30991d891b
I20260806 09:02:12.626140 17092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: Bootstrap starting.
I20260806 09:02:12.629571 17092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:12.630825 17092 log.cc:824] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:12.631739 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:12.619635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.22.98.62:37713)
0806 09:02:12.620764 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:12.620777 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:12.620823 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:02:12.622514 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:12.622541 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:12.623064 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:12.623477 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:12.624120 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:12.625416 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:12.626863 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:12.627623 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:12.627824 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:12.628991 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:12.629000 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:12.629016 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:12.629571 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:12.629683 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:12.629706 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:12.629832 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:12.630136 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 09:02:12.630488 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":375,"threads_started":1}
I20260806 09:02:12.634482 17092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: No bootstrap required, opened a new log
I20260806 09:02:12.642318 17092 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } }
I20260806 09:02:12.642748 17092 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:12.642844 17092 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d938a5f99e4b508e5df91ebb45df51, State: Initialized, Role: FOLLOWER
I20260806 09:02:12.643440 17092 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } }
I20260806 09:02:12.643746 17092 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:12.643862 17092 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:12.644017 17092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:12.646685 17092 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } }
I20260806 09:02:12.647205 17092 leader_election.cc:302] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0d938a5f99e4b508e5df91ebb45df51; no voters:
I20260806 09:02:12.647822 17092 leader_election.cc:288] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:12.647976 17100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:12.648774 17100 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [term 1 LEADER]: Becoming Leader. State: Replica: a0d938a5f99e4b508e5df91ebb45df51, State: Running, Role: LEADER
I20260806 09:02:12.649304 17100 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } }
I20260806 09:02:12.650066 17092 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:12.652298 17102 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0d938a5f99e4b508e5df91ebb45df51. Latest consensus state: current_term: 1 leader_uuid: "a0d938a5f99e4b508e5df91ebb45df51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } } }
I20260806 09:02:12.652482 17101 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0d938a5f99e4b508e5df91ebb45df51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d938a5f99e4b508e5df91ebb45df51" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37713 } } }
I20260806 09:02:12.652781 17102 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:12.652796 17101 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:12.654847 17113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:12.660346 17113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:12.670379 17113 catalog_manager.cc:1347] Generated new cluster ID: 20d92381012e4459aa44e6c180c07fed
I20260806 09:02:12.670524 17113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:12.682432 17113 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:12.683823 17113 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:12.692494 17113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: Generated new TSK 0
I20260806 09:02:12.693404 17113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:12.705283 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--builtin_ntp_servers=127.22.98.20:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:13.075743 17133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:13.076099 17133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:13.076251 17133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:13.084470 17133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:13.084625 17133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:13.084781 17133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:13.094954 17133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:13.096479 17133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:13.098498 17133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:13.113353 17144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:13.113328 17142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:13.113286 17150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:13.114156 17133 server_base.cc:1034] running on GCE node
I20260806 09:02:13.114814 17133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:13.115901 17133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:13.117130 17133 hybrid_clock.cc:648] HybridClock initialized: now 1786006933117062 us; error 58 us; skew 500 ppm
I20260806 09:02:13.117496 17133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:13.120085 17133 webserver.cc:460] Webserver started at http://127.22.98.1:37485/ using document root <none> and password file <none>
I20260806 09:02:13.120812 17133 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:13.120911 17133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:13.121198 17133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:13.124444 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "c8e92e1c7d9340d7b575c990d872f4a9"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "a35bce7188cdfd8ca0fee41d44c032d3"
server_key_iv: "c48acc7f89c1f7c1813d9a13d608fd0d"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.125279 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "c8e92e1c7d9340d7b575c990d872f4a9"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "a35bce7188cdfd8ca0fee41d44c032d3"
server_key_iv: "c48acc7f89c1f7c1813d9a13d608fd0d"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.131000 17133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:02:13.134965 17156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.136656 17133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:13.136868 17133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "c8e92e1c7d9340d7b575c990d872f4a9"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "a35bce7188cdfd8ca0fee41d44c032d3"
server_key_iv: "c48acc7f89c1f7c1813d9a13d608fd0d"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.137118 17133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:13.154441 17133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:13.155298 17133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:13.155603 17133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:13.156589 17133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:13.158106 17133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:13.158195 17133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.158313 17133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:13.158389 17133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.189706 17133 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:42315
I20260806 09:02:13.189760 17317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:42315 every 8 connection(s)
I20260806 09:02:13.191164 17133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:13.193238 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 17133
I20260806 09:02:13.193419 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:13.194085 22920 external_mini_cluster.cc:1428] Setting key 8971e45ba2e7d7a68ad4ce376eea18f9
I20260806 09:02:13.196996 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--builtin_ntp_servers=127.22.98.20:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:13.204397 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:13.193938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:58133 (local address 127.22.98.62:37713)
0806 09:02:13.194377 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:13.194381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:13.197895 (+ 3514us) server_negotiation.cc:407] Connection header received
0806 09:02:13.197955 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:13.197958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:13.198033 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:13.198125 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:13.199274 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.200030 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.201579 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.201999 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.202111 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:13.203494 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:13.203502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:13.203505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:13.203525 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:13.203606 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:13.203624 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:13.203702 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:13.203921 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 09:02:13.204026 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":246,"threads_started":1}
I20260806 09:02:13.205428 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:13.194651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37713 (local address 127.22.98.1:58133)
0806 09:02:13.195787 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:13.195834 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:13.197591 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:13.198415 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:13.198424 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:13.198518 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:13.199098 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:13.199116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.200187 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.200192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:13.201430 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:13.201444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.202233 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.202242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:13.202468 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:13.203148 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:13.203151 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:13.203192 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:13.203304 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:13.203714 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:13.203726 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:13.203736 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 09:02:13.203896 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 09:02:13.204325 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260806 09:02:13.206948 17319 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37713
I20260806 09:02:13.207293 17319 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:13.208278 17319 heartbeater.cc:511] Master 127.22.98.62:37713 requested a full tablet report, sending...
I20260806 09:02:13.210599 17040 ts_manager.cc:194] Registered new tserver with Master: c8e92e1c7d9340d7b575c990d872f4a9 (127.22.98.1:42315)
I20260806 09:02:13.212615 17040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.1:58133
W20260806 09:02:13.549110 17326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:13.549562 17326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:13.549693 17326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:13.558290 17326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:13.558455 17326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:13.558655 17326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:13.568993 17326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:13.570436 17326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:13.572402 17326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:13.585739 17331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:13.585655 17336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:13.585731 17333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:13.586578 17326 server_base.cc:1034] running on GCE node
I20260806 09:02:13.587204 17326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:13.588238 17326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:13.589462 17326 hybrid_clock.cc:648] HybridClock initialized: now 1786006933589399 us; error 52 us; skew 500 ppm
I20260806 09:02:13.589814 17326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:13.592527 17326 webserver.cc:460] Webserver started at http://127.22.98.2:37561/ using document root <none> and password file <none>
I20260806 09:02:13.593302 17326 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:13.593405 17326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:13.593729 17326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:13.596869 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "cba8ccb468905a6ce402950f1e8c9f59"
server_key_iv: "b5da3b7f397acc1ff67e2d8a41991e38"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.597698 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "cba8ccb468905a6ce402950f1e8c9f59"
server_key_iv: "b5da3b7f397acc1ff67e2d8a41991e38"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.603330 17326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:02:13.607285 17345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.608844 17326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:13.609015 17326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d"
format_stamp: "Formatted at 2026-08-06 09:02:13 on dist-test-slave-jpj1"
server_key: "cba8ccb468905a6ce402950f1e8c9f59"
server_key_iv: "b5da3b7f397acc1ff67e2d8a41991e38"
server_key_version: "encryptionkey@0"
I20260806 09:02:13.609234 17326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:13.628381 17326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:13.629311 17326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:13.629613 17326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:13.630725 17326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:13.632395 17326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:13.632499 17326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.632614 17326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:13.632699 17326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:13.665130 17326 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:40221
I20260806 09:02:13.665186 17507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:40221 every 8 connection(s)
I20260806 09:02:13.666522 17326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:13.671943 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 17326
I20260806 09:02:13.672132 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:13.672695 22920 external_mini_cluster.cc:1428] Setting key e182e69e42ba7046ce28bf2534a6b573
I20260806 09:02:13.675967 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--builtin_ntp_servers=127.22.98.20:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:13.681255 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:13.669311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:59167 (local address 127.22.98.62:37713)
0806 09:02:13.669495 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:13.669500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:13.671025 (+ 1525us) server_negotiation.cc:407] Connection header received
0806 09:02:13.673449 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:13.673456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:13.673549 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:13.673654 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:13.674914 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.675793 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.678129 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.678572 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.678686 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:13.680420 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:13.680426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:13.680430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:13.680452 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:13.680489 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:13.680503 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:13.680587 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:13.680884 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 09:02:13.680986 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:02:13.682644 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:13.669875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37713 (local address 127.22.98.2:59167)
0806 09:02:13.670857 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:13.670901 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:13.673102 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:13.673938 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:13.673947 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:13.674041 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:13.674718 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:13.674740 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.675964 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.675970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:13.677958 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:13.677974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:13.678822 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:13.678828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:13.679118 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:13.680001 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:13.680004 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:13.680062 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:13.680182 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:13.680600 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:13.680611 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:13.680619 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 09:02:13.680824 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 09:02:13.681273 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":319,"threads_started":1}
I20260806 09:02:13.684231 17508 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37713
I20260806 09:02:13.684593 17508 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:13.685575 17508 heartbeater.cc:511] Master 127.22.98.62:37713 requested a full tablet report, sending...
I20260806 09:02:13.687269 17040 ts_manager.cc:194] Registered new tserver with Master: fec5ef7b25974ca08e1f4ddf0c0d645d (127.22.98.2:40221)
I20260806 09:02:13.688282 17040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.2:59167
W20260806 09:02:14.026412 17511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:14.026711 17511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:14.026824 17511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:14.034212 17511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:14.034332 17511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:14.034469 17511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:14.043967 17511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:14.045346 17511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:14.047227 17511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:14.060433 17517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:14.060385 17518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:14.060456 17521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:14.061347 17511 server_base.cc:1034] running on GCE node
I20260806 09:02:14.061985 17511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:14.062999 17511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:14.064232 17511 hybrid_clock.cc:648] HybridClock initialized: now 1786006934064178 us; error 67 us; skew 500 ppm
I20260806 09:02:14.064589 17511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:14.071187 17511 webserver.cc:460] Webserver started at http://127.22.98.3:43701/ using document root <none> and password file <none>
I20260806 09:02:14.072049 17511 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:14.072173 17511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:14.072516 17511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:14.075798 17511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "30e9e9d0d6794048ab6d3d72d4f974e5"
format_stamp: "Formatted at 2026-08-06 09:02:14 on dist-test-slave-jpj1"
server_key: "18cb20aca4764c11329f25b55679d097"
server_key_iv: "5d429102a3b64b3ce95d640b10f2c1b2"
server_key_version: "encryptionkey@0"
I20260806 09:02:14.076660 17511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "30e9e9d0d6794048ab6d3d72d4f974e5"
format_stamp: "Formatted at 2026-08-06 09:02:14 on dist-test-slave-jpj1"
server_key: "18cb20aca4764c11329f25b55679d097"
server_key_iv: "5d429102a3b64b3ce95d640b10f2c1b2"
server_key_version: "encryptionkey@0"
I20260806 09:02:14.082502 17511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:02:14.086539 17527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:14.088220 17511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:14.088424 17511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "30e9e9d0d6794048ab6d3d72d4f974e5"
format_stamp: "Formatted at 2026-08-06 09:02:14 on dist-test-slave-jpj1"
server_key: "18cb20aca4764c11329f25b55679d097"
server_key_iv: "5d429102a3b64b3ce95d640b10f2c1b2"
server_key_version: "encryptionkey@0"
I20260806 09:02:14.088652 17511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:14.101111 17511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:14.101990 17511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:14.102284 17511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:14.103662 17511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:14.105363 17511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:14.105468 17511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:14.105581 17511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:14.105654 17511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:14.137408 17511 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:34757
I20260806 09:02:14.137459 17682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:34757 every 8 connection(s)
I20260806 09:02:14.138736 17511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:14.140007 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 17511
I20260806 09:02:14.140203 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:14.140625 22920 external_mini_cluster.cc:1428] Setting key 32e10a868e5c663b18b50f9f7c53fabd
I20260806 09:02:14.152712 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.142845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:44885 (local address 127.22.98.62:37713)
0806 09:02:14.143050 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.143057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.144650 (+ 1593us) server_negotiation.cc:407] Connection header received
0806 09:02:14.146392 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.146396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.146465 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.146546 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:14.147606 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.148430 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.150000 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.150466 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.150603 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.151963 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:14.151969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:14.151972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:14.151989 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:14.152021 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:14.152033 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:14.152096 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.152313 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.152398 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:02:14.153710 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.143490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37713 (local address 127.22.98.3:44885)
0806 09:02:14.144507 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.144545 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.146137 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.146731 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.146739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.146866 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:14.147430 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.147447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.148581 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.148585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.149870 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.149882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.150679 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.150685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.150925 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.151605 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:14.151608 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:14.151665 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:14.151755 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:14.152102 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:14.152108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:14.152114 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.152252 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.152538 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":334,"threads_started":1}
I20260806 09:02:14.155320 17683 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37713
I20260806 09:02:14.155670 17683 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:14.156499 17683 heartbeater.cc:511] Master 127.22.98.62:37713 requested a full tablet report, sending...
I20260806 09:02:14.157970 17036 ts_manager.cc:194] Registered new tserver with Master: 30e9e9d0d6794048ab6d3d72d4f974e5 (127.22.98.3:34757)
I20260806 09:02:14.158844 17036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.3:44885
I20260806 09:02:14.168757 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:14.178648 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.172833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.22.98.62:37713)
0806 09:02:14.173056 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.173063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.173373 (+ 310us) server_negotiation.cc:407] Connection header received
0806 09:02:14.173510 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.173514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.173579 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.173658 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:14.174121 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.174823 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.176262 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.176729 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.176838 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.177958 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:14.177963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:14.177967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:14.177985 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:14.178020 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:14.178032 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:14.178140 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.178267 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.178318 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:02:14.182852 17036 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:14.189894 17036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:14.216593 17319 heartbeater.cc:503] Master 127.22.98.62:37713 was elected leader, sending a full tablet report...
I20260806 09:02:14.225154 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.203156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:40221 (local address 127.0.0.1:48124)
0806 09:02:14.213937 (+ 10781us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.214010 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.214273 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.216401 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.216410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.216442 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:14.216926 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.216942 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.218985 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.218990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.221548 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.221564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.223450 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.223457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.224216 (+ 759us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.224293 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.224427 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.224640 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10410,"mutex_wait_us":76,"thread_start_us":15664,"threads_started":1}
I20260806 09:02:14.225550 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.203169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48124 (local address 127.22.98.2:40221)
0806 09:02:14.215639 (+ 12470us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.215648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.215683 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:02:14.215813 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.215822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.216098 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.217174 (+ 1076us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:14.217409 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.218824 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.221762 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.223228 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.223366 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.223806 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.225053 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.225238 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12278,"thread_start_us":12179,"threads_started":1}
I20260806 09:02:14.227435 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.203728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39452)
0806 09:02:14.218665 (+ 14937us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.218693 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.218946 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.220030 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.220035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.220061 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:14.220703 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.220714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.222543 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.222549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.224680 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.224705 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.226651 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.226658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.226817 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.226852 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.226945 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.227073 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14806,"mutex_wait_us":28,"thread_start_us":10787,"threads_started":1}
I20260806 09:02:14.227588 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.204460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.22.98.1:42315)
0806 09:02:14.211130 (+ 6670us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.211142 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.218823 (+ 7681us) server_negotiation.cc:407] Connection header received
0806 09:02:14.219195 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.219204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.219512 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.219739 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:14.220959 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.222309 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.224935 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.226432 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.226587 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.226873 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.227007 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.227192 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6448,"thread_start_us":6287,"threads_started":1}
I20260806 09:02:14.227923 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.203381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:34757 (local address 127.0.0.1:57826)
0806 09:02:14.215014 (+ 11633us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.215040 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.215173 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.220219 (+ 5046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.220223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.220252 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:14.221176 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.221188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.223059 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.223065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.225337 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.225352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.227181 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.227188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.227373 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.227407 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.227478 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.227637 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11494,"thread_start_us":11298,"threads_started":1}
I20260806 09:02:14.228756 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.203946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.22.98.3:34757)
0806 09:02:14.219642 (+ 15696us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.219650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.219686 (+ 36us) server_negotiation.cc:407] Connection header received
0806 09:02:14.219789 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.219798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.220034 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.220228 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:14.221366 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.222904 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.225568 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.226965 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.227929 (+ 964us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.228197 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.228282 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.228442 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15510,"thread_start_us":15347,"threads_started":1}
I20260806 09:02:14.230158 17419 tablet_service.cc:1467] Processing CreateTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce (DEFAULT_TABLE table=test-table [id=a10a86ff51b44298bd4928f1246654de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:14.231351 17224 tablet_service.cc:1467] Processing CreateTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce (DEFAULT_TABLE table=test-table [id=a10a86ff51b44298bd4928f1246654de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:14.231846 17419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72255cf8e4b34bb58a786d8e4b11c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:14.232564 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72255cf8e4b34bb58a786d8e4b11c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:14.232520 17592 tablet_service.cc:1467] Processing CreateTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce (DEFAULT_TABLE table=test-table [id=a10a86ff51b44298bd4928f1246654de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:14.233958 17592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72255cf8e4b34bb58a786d8e4b11c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:14.244774 17706 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Bootstrap starting.
I20260806 09:02:14.247015 17707 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Bootstrap starting.
I20260806 09:02:14.250294 17706 tablet_bootstrap.cc:654] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:14.251022 17707 tablet_bootstrap.cc:654] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:14.251879 17706 log.cc:824] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:14.252324 17707 log.cc:824] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:14.254738 17708 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Bootstrap starting.
I20260806 09:02:14.256091 17707 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: No bootstrap required, opened a new log
I20260806 09:02:14.256425 17707 ts_tablet_manager.cc:1397] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 09:02:14.258258 17708 tablet_bootstrap.cc:654] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:14.259315 17708 log.cc:824] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:14.262331 17708 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: No bootstrap required, opened a new log
I20260806 09:02:14.262589 17706 tablet_bootstrap.cc:492] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: No bootstrap required, opened a new log
I20260806 09:02:14.262611 17708 ts_tablet_manager.cc:1397] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 09:02:14.262881 17706 ts_tablet_manager.cc:1397] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.004s
I20260806 09:02:14.267297 17707 raft_consensus.cc:348] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.268019 17707 raft_consensus.cc:374] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:14.268189 17707 raft_consensus.cc:729] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec5ef7b25974ca08e1f4ddf0c0d645d, State: Initialized, Role: FOLLOWER
I20260806 09:02:14.269068 17707 consensus_queue.cc:260] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.270386 17508 heartbeater.cc:503] Master 127.22.98.62:37713 was elected leader, sending a full tablet report...
I20260806 09:02:14.270421 17708 raft_consensus.cc:348] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.270901 17708 raft_consensus.cc:374] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:14.271057 17708 raft_consensus.cc:729] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e9e9d0d6794048ab6d3d72d4f974e5, State: Initialized, Role: FOLLOWER
I20260806 09:02:14.272011 17707 ts_tablet_manager.cc:1428] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260806 09:02:14.271899 17708 consensus_queue.cc:260] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.273037 17683 heartbeater.cc:503] Master 127.22.98.62:37713 was elected leader, sending a full tablet report...
I20260806 09:02:14.272987 17706 raft_consensus.cc:348] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.273451 17706 raft_consensus.cc:374] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:14.273622 17706 raft_consensus.cc:729] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e92e1c7d9340d7b575c990d872f4a9, State: Initialized, Role: FOLLOWER
I20260806 09:02:14.274346 17708 ts_tablet_manager.cc:1428] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 09:02:14.274324 17706 consensus_queue.cc:260] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.278059 17706 ts_tablet_manager.cc:1428] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260806 09:02:14.391947 17684 tablet.cc:2369] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:14.418790 17509 tablet.cc:2369] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:14.443521 17320 tablet.cc:2369] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:14.500133 17713 raft_consensus.cc:482] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:14.500478 17713 raft_consensus.cc:504] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.502317 17713 leader_election.cc:288] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec5ef7b25974ca08e1f4ddf0c0d645d (127.22.98.2:40221), c8e92e1c7d9340d7b575c990d872f4a9 (127.22.98.1:42315)
I20260806 09:02:14.508128 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.502659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.22.98.3:48869)
0806 09:02:14.502945 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.502966 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.503095 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.503592 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.503597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.503623 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:14.503775 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.503784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.504791 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.504795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.506166 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.506182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.507313 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.507319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.507502 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.507526 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.507658 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.507826 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":27}
I20260806 09:02:14.508111 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.502918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:48869 (local address 127.22.98.1:42315)
0806 09:02:14.503083 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.503089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.503110 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:02:14.503223 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.503228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.503306 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.503381 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:14.503947 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.504645 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.506338 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.507144 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.507265 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.507379 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.507712 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.507852 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:02:14.508828 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.502864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:54775 (local address 127.22.98.2:40221)
0806 09:02:14.503081 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.503088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.503773 (+ 685us) server_negotiation.cc:407] Connection header received
0806 09:02:14.503951 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.503956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.504028 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.504106 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:14.504627 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.505299 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.507064 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.507897 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.508014 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.508108 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.508453 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.508586 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 09:02:14.509529 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.502723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:40221 (local address 127.22.98.3:54775)
0806 09:02:14.503655 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.503672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:14.503817 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:14.504135 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:14.504140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:14.504164 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:14.504476 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.504485 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.505445 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.505449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.506869 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:14.506883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.508066 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.508072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:14.508266 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.508302 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:02:14.509036 (+ 734us) client_negotiation.cc:240] Negotiation successful
0806 09:02:14.509176 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":1,"thread_start_us":529,"threads_started":1}
I20260806 09:02:14.509346 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72255cf8e4b34bb58a786d8e4b11c6ce" candidate_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" is_pre_election: true
I20260806 09:02:14.510032 17250 raft_consensus.cc:2454] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e9e9d0d6794048ab6d3d72d4f974e5 in term 0.
I20260806 09:02:14.510704 17530 leader_election.cc:302] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e9e9d0d6794048ab6d3d72d4f974e5, c8e92e1c7d9340d7b575c990d872f4a9; no voters:
I20260806 09:02:14.510649 17447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72255cf8e4b34bb58a786d8e4b11c6ce" candidate_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" is_pre_election: true
I20260806 09:02:14.511094 17713 raft_consensus.cc:2781] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:14.511133 17447 raft_consensus.cc:2454] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e9e9d0d6794048ab6d3d72d4f974e5 in term 0.
I20260806 09:02:14.511219 17713 raft_consensus.cc:482] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:14.511314 17713 raft_consensus.cc:3037] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:14.513835 17713 raft_consensus.cc:504] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.514525 17713 leader_election.cc:288] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [CANDIDATE]: Term 1 election: Requested vote from peers fec5ef7b25974ca08e1f4ddf0c0d645d (127.22.98.2:40221), c8e92e1c7d9340d7b575c990d872f4a9 (127.22.98.1:42315)
I20260806 09:02:14.515056 17447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72255cf8e4b34bb58a786d8e4b11c6ce" candidate_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d"
I20260806 09:02:14.515252 17447 raft_consensus.cc:3037] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:14.515440 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72255cf8e4b34bb58a786d8e4b11c6ce" candidate_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8e92e1c7d9340d7b575c990d872f4a9"
I20260806 09:02:14.515689 17250 raft_consensus.cc:3037] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:14.518611 17447 raft_consensus.cc:2454] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e9e9d0d6794048ab6d3d72d4f974e5 in term 1.
I20260806 09:02:14.518796 17250 raft_consensus.cc:2454] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e9e9d0d6794048ab6d3d72d4f974e5 in term 1.
I20260806 09:02:14.519227 17529 leader_election.cc:302] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e9e9d0d6794048ab6d3d72d4f974e5, fec5ef7b25974ca08e1f4ddf0c0d645d; no voters:
I20260806 09:02:14.519598 17713 raft_consensus.cc:2781] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:14.520164 17713 raft_consensus.cc:686] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 1 LEADER]: Becoming Leader. State: Replica: 30e9e9d0d6794048ab6d3d72d4f974e5, State: Running, Role: LEADER
I20260806 09:02:14.520763 17713 consensus_queue.cc:237] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } }
I20260806 09:02:14.525955 17036 catalog_manager.cc:5495] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30e9e9d0d6794048ab6d3d72d4f974e5 (127.22.98.3). New cstate: current_term: 1 leader_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30e9e9d0d6794048ab6d3d72d4f974e5" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 34757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 } health_report { overall_health: UNKNOWN } } }
I20260806 09:02:14.598445 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.592699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.22.98.3:34757)
0806 09:02:14.592893 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.592899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.592983 (+ 84us) server_negotiation.cc:407] Connection header received
0806 09:02:14.593261 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.593268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.593379 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.593562 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:14.593999 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.594733 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.596244 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.596669 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.596766 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.597236 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:14.597795 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:14.597938 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.598095 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.598141 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 09:02:14.606920 17250 raft_consensus.cc:1261] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 1 FOLLOWER]: Refusing update from remote peer 30e9e9d0d6794048ab6d3d72d4f974e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:14.607053 17447 raft_consensus.cc:1261] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 1 FOLLOWER]: Refusing update from remote peer 30e9e9d0d6794048ab6d3d72d4f974e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:14.608026 17717 consensus_queue.cc:1035] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec5ef7b25974ca08e1f4ddf0c0d645d" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 40221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:14.608467 17713 consensus_queue.cc:1035] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8e92e1c7d9340d7b575c990d872f4a9" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 42315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:14.641149 17040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:14.641458 17040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:14.645334 17040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0d938a5f99e4b508e5df91ebb45df51: Sending DeleteTablet for 3 replicas of tablet 72255cf8e4b34bb58a786d8e4b11c6ce
I20260806 09:02:14.646631 17419 tablet_service.cc:1514] Processing DeleteTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:14 UTC) from {username='slave'} at 127.0.0.1:48124
I20260806 09:02:14.646637 17592 tablet_service.cc:1514] Processing DeleteTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:14 UTC) from {username='slave'} at 127.0.0.1:57826
I20260806 09:02:14.647416 17224 tablet_service.cc:1514] Processing DeleteTablet for tablet 72255cf8e4b34bb58a786d8e4b11c6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:14 UTC) from {username='slave'} at 127.0.0.1:39452
I20260806 09:02:14.649799 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 17133
I20260806 09:02:14.673250 17739 tablet_replica.cc:330] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: stopping tablet replica
I20260806 09:02:14.674086 17739 raft_consensus.cc:2229] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:14.674932 17739 raft_consensus.cc:2258] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:14.676872 17737 tablet_replica.cc:330] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: stopping tablet replica
I20260806 09:02:14.677304 17736 tablet_replica.cc:330] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: stopping tablet replica
I20260806 09:02:14.677829 17737 raft_consensus.cc:2229] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:14.678332 17736 raft_consensus.cc:2229] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:14.679055 17736 raft_consensus.cc:2258] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:14.679625 17737 raft_consensus.cc:2258] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:14.682857 17737 ts_tablet_manager.cc:1905] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:14.684248 17739 ts_tablet_manager.cc:1905] T 72255cf8e4b34bb58a786d8e4b11c6ce P c8e92e1c7d9340d7b575c990d872f4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:14.684247 17736 ts_tablet_manager.cc:1905] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:14.685127 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.651446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.22.98.1:42315)
0806 09:02:14.651663 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.651670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.671158 (+ 19488us) server_negotiation.cc:407] Connection header received
0806 09:02:14.671398 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.671404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.671574 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.671719 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:14.676885 (+ 5166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.678010 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.680641 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.681305 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.681422 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.683179 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:14.683193 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:14.683212 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:14.684080 (+ 868us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:14.684156 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:14.684190 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:14.684319 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.684603 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.684736 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:02:14.686585 17309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:14.692751 17736 ts_tablet_manager.cc:1918] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:14.697363 17736 log.cc:1196] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/72255cf8e4b34bb58a786d8e4b11c6ce
I20260806 09:02:14.698103 17736 ts_tablet_manager.cc:1939] T 72255cf8e4b34bb58a786d8e4b11c6ce P 30e9e9d0d6794048ab6d3d72d4f974e5: Deleting consensus metadata
I20260806 09:02:14.700480 17020 catalog_manager.cc:4841] TS 30e9e9d0d6794048ab6d3d72d4f974e5 (127.22.98.3:34757): tablet 72255cf8e4b34bb58a786d8e4b11c6ce (table test-table [id=a10a86ff51b44298bd4928f1246654de]) successfully deleted
I20260806 09:02:14.700569 17737 ts_tablet_manager.cc:1918] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:14.700763 17737 log.cc:1196] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/72255cf8e4b34bb58a786d8e4b11c6ce
I20260806 09:02:14.701570 17737 ts_tablet_manager.cc:1939] T 72255cf8e4b34bb58a786d8e4b11c6ce P fec5ef7b25974ca08e1f4ddf0c0d645d: Deleting consensus metadata
I20260806 09:02:14.704468 17014 catalog_manager.cc:4841] TS fec5ef7b25974ca08e1f4ddf0c0d645d (127.22.98.2:40221): tablet 72255cf8e4b34bb58a786d8e4b11c6ce (table test-table [id=a10a86ff51b44298bd4928f1246654de]) successfully deleted
I20260806 09:02:14.791688 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 17133
I20260806 09:02:14.819623 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 17326
W20260806 09:02:14.819638 17015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.98.1:42315 (error 108)
I20260806 09:02:14.822238 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.821330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39468)
0806 09:02:14.821906 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.822031 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":358,"threads_started":1}
W20260806 09:02:14.822543 17015 catalog_manager.cc:4584] TS c8e92e1c7d9340d7b575c990d872f4a9 (127.22.98.1:42315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72255cf8e4b34bb58a786d8e4b11c6ce: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
I20260806 09:02:14.827368 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.820488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.22.98.2:40221)
0806 09:02:14.820681 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.820688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.821144 (+ 456us) server_negotiation.cc:407] Connection header received
0806 09:02:14.821232 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.821239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.821360 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.821466 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:14.821976 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.822811 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.824360 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.824878 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.824996 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.826011 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:14.826021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:14.826036 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:14.826626 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:14.826682 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:14.826702 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:14.826801 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.826934 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.827044 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:02:14.828038 17500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:14.876369 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.875787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39474)
0806 09:02:14.876018 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.876144 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 09:02:14.877622 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.877195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39478)
0806 09:02:14.877348 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.877454 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 09:02:14.918001 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 17326
I20260806 09:02:14.938419 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.937809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39494)
0806 09:02:14.938078 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.938197 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 09:02:14.939764 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.939248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39504)
0806 09:02:14.939439 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:14.939560 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 09:02:14.942512 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 17511
I20260806 09:02:14.950729 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:14.943415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57840 (local address 127.22.98.3:34757)
0806 09:02:14.943654 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:14.943660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:14.943683 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:02:14.943768 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:14.943775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:14.943917 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:14.944063 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:14.944500 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.945724 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.946999 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:14.947706 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:14.947856 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:14.948855 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:14.948869 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:14.948892 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:14.949777 (+ 885us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:14.949851 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:14.949879 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:14.949993 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:14.950131 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 09:02:14.950240 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:02:14.951582 17676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:15.020606 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:15.020099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39514)
0806 09:02:15.020336 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:15.020448 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 09:02:15.021672 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:15.021250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39530)
0806 09:02:15.021458 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:15.021530 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 09:02:15.074813 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 17511
I20260806 09:02:15.107841 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 16992
I20260806 09:02:15.109166 17086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:15.219138 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 16992
I20260806 09:02:15.219223 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:15.218524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:42315 (local address 127.0.0.1:39544)
0806 09:02:15.218899 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:15.219051 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.1:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
2026-08-06T09:02:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3181 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:15 dist-test-slave-jpj1 krb5kdc[17753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:15 dist-test-slave-jpj1 krb5kdc[17753](info): set up 2 sockets
Aug 06 09:02:15 dist-test-slave-jpj1 krb5kdc[17753](info): commencing operation
krb5kdc: starting...
W20260806 09:02:16.429805 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:16 dist-test-slave-jpj1 krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:16.761336 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37657
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:17.116930 17771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:17.117259 17771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:17.117377 17771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:17.124805 17771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:17.124929 17771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:17.124990 17771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:17.125073 17771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:17.125131 17771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:17.135514 17771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:17.137002 17771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:17.138382 17771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:17.150542 17782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:17.150476 17783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:17.150470 17785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:17.151357 17771 server_base.cc:1034] running on GCE node
I20260806 09:02:17.152223 17771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:17.153687 17771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:17.154899 17771 hybrid_clock.cc:648] HybridClock initialized: now 1786006937154846 us; error 42 us; skew 500 ppm
I20260806 09:02:17.155249 17771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:17 dist-test-slave-jpj1 krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006937, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:17.158420 17771 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:17.160104 17771 webserver.cc:460] Webserver started at http://127.22.98.62:34221/ using document root <none> and password file <none>
I20260806 09:02:17.160885 17771 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:17.161002 17771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:17.161329 17771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:17.164605 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "181a117204fb4d06bcf84de160795e46"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "cb1d0b34b4998d2eda71f30e3159705b"
server_key_iv: "203666ce4a575a554cd4f00474ace3c3"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.165409 17771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "181a117204fb4d06bcf84de160795e46"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "cb1d0b34b4998d2eda71f30e3159705b"
server_key_iv: "203666ce4a575a554cd4f00474ace3c3"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.170976 17771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 09:02:17.174958 17795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:17.176604 17771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:17.176823 17771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "181a117204fb4d06bcf84de160795e46"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "cb1d0b34b4998d2eda71f30e3159705b"
server_key_iv: "203666ce4a575a554cd4f00474ace3c3"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.177058 17771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:17.196878 17771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:17.201598 17771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:17.201929 17771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:17.220916 17771 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37657
I20260806 09:02:17.220969 17867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37657 every 8 connection(s)
I20260806 09:02:17.222398 17771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:17.223779 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 17771
I20260806 09:02:17.223975 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:17.224515 22920 external_mini_cluster.cc:1428] Setting key e137211e9eb3a704f05bd9241b735a71
I20260806 09:02:17.227399 17868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:02:17 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:17.239307 17868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: Bootstrap starting.
I20260806 09:02:17.242763 17868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:17.243937 17868 log.cc:824] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:17.247516 17868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: No bootstrap required, opened a new log
I20260806 09:02:17.247979 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:17.226572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.22.98.62:37657)
0806 09:02:17.227418 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:17.227431 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:17.227481 (+ 50us) server_negotiation.cc:407] Connection header received
0806 09:02:17.229277 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:17.229300 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:17.229719 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:17.230214 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:17.231832 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.233269 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.234671 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.235407 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.235628 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:17.240253 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:17.240263 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:17.240280 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:17.240328 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:17.243469 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:17.244321 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:17.244329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:17.244336 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:17.244593 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:17.245043 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:17.245049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:17.245052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:17.245674 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:17.245928 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:17.246443 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 09:02:17.246846 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":358,"threads_started":1}
I20260806 09:02:17.255424 17868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } }
I20260806 09:02:17.255906 17868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:17.256009 17868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181a117204fb4d06bcf84de160795e46, State: Initialized, Role: FOLLOWER
I20260806 09:02:17.256650 17868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } }
I20260806 09:02:17.256893 17868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:17.257057 17868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:17.257277 17868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:17.259763 17868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } }
I20260806 09:02:17.260372 17868 leader_election.cc:302] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 181a117204fb4d06bcf84de160795e46; no voters:
I20260806 09:02:17.261063 17868 leader_election.cc:288] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:17.261271 17873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:17.262004 17873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [term 1 LEADER]: Becoming Leader. State: Replica: 181a117204fb4d06bcf84de160795e46, State: Running, Role: LEADER
I20260806 09:02:17.262709 17873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } }
I20260806 09:02:17.263478 17868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:17.265976 17875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 181a117204fb4d06bcf84de160795e46. Latest consensus state: current_term: 1 leader_uuid: "181a117204fb4d06bcf84de160795e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } } }
I20260806 09:02:17.266448 17875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:17.266918 17874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "181a117204fb4d06bcf84de160795e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181a117204fb4d06bcf84de160795e46" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37657 } } }
I20260806 09:02:17.267277 17874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:17.268196 17881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:17.273792 17881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:17.283584 17881 catalog_manager.cc:1347] Generated new cluster ID: a8ec8e880d4047d7b6456335b3efc571
I20260806 09:02:17.283720 17881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:17.298817 17881 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:17.300253 17881 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:17.312625 17881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: Generated new TSK 0
I20260806 09:02:17.313552 17881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.98.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
Entry for principal kudu/127.22.98.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab.
I20260806 09:02:17.395308 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:17.764680 17907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:17.765062 17907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:17.765208 17907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:17.773031 17907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:17.773187 17907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:17.773346 17907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:17.783841 17907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.1.keytab
--principal=kudu/127.22.98.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:17.785281 17907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:17.786718 17907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:17.800952 17914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:17.801008 17915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:17.800952 17919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:17.801673 17907 server_base.cc:1034] running on GCE node
I20260806 09:02:17.802361 17907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:17.803329 17907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:17.804563 17907 hybrid_clock.cc:648] HybridClock initialized: now 1786006937804493 us; error 55 us; skew 500 ppm
I20260806 09:02:17.804934 17907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:17 dist-test-slave-jpj1 krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006937, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:17.808373 17907 init.cc:377] Logged in from keytab as kudu/127.22.98.1@KRBTEST.COM (short username kudu)
I20260806 09:02:17.810103 17907 webserver.cc:460] Webserver started at http://127.22.98.1:33153/ using document root <none> and password file <none>
I20260806 09:02:17.810835 17907 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:17.810966 17907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:17.811299 17907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:17.814582 17907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "2ef72cab1fe84f70a85602c9a7402be1"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "b4a9df73adf4ac21362e57e0157335da"
server_key_iv: "5617e785b0c0d88ca65d24415b3f4184"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.815393 17907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "2ef72cab1fe84f70a85602c9a7402be1"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "b4a9df73adf4ac21362e57e0157335da"
server_key_iv: "5617e785b0c0d88ca65d24415b3f4184"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.821126 17907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 09:02:17.825248 17930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:17.826753 17907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:17.826946 17907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "2ef72cab1fe84f70a85602c9a7402be1"
format_stamp: "Formatted at 2026-08-06 09:02:17 on dist-test-slave-jpj1"
server_key: "b4a9df73adf4ac21362e57e0157335da"
server_key_iv: "5617e785b0c0d88ca65d24415b3f4184"
server_key_version: "encryptionkey@0"
I20260806 09:02:17.827155 17907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:17.853343 17907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:17.858013 17907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:17.858336 17907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:17.859447 17907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:17.861128 17907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:17.861227 17907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:17.861368 17907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:17.861426 17907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:17.893942 17907 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:44981
I20260806 09:02:17.893990 18096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:44981 every 8 connection(s)
I20260806 09:02:17.895406 17907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:17.900817 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 17907
I20260806 09:02:17.901028 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:17.901640 22920 external_mini_cluster.cc:1428] Setting key 9e83f55987de860b1c047dca3f591ff0
Aug 06 09:02:17 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006937, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.1@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:17.917707 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:17.898079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:45355 (local address 127.22.98.62:37657)
0806 09:02:17.898553 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:17.898558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:17.899521 (+ 963us) server_negotiation.cc:407] Connection header received
0806 09:02:17.902085 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:17.902118 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:17.902204 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:17.902335 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:17.904105 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.904821 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.907029 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.907458 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.907606 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:17.911778 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:17.911786 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:17.911790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:17.911828 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:17.914308 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:17.915202 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:17.915205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:17.915208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:17.915343 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:17.915843 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:17.915847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:17.915849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:17.916134 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:17.916250 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:17.917172 (+ 922us) server_negotiation.cc:299] Negotiation successful
0806 09:02:17.917378 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260806 09:02:17.918798 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:17.898561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37657 (local address 127.22.98.1:45355)
0806 09:02:17.899348 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:17.899391 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:17.901697 (+ 2306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:17.902569 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:17.902579 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:17.903157 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:17.903889 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:17.903919 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.905005 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.905011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:17.906862 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:17.906877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:17.907714 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:17.907722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:17.908003 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:17.908734 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:17.908770 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:17.911497 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:17.914465 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:17.914475 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:17.914493 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:17.915096 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:17.915443 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:17.915447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:17.915450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:17.915732 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:17.916265 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:17.916272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:17.916729 (+ 457us) client_negotiation.cc:769] Sending connection context
0806 09:02:17.917048 (+ 319us) client_negotiation.cc:240] Negotiation successful
0806 09:02:17.917630 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":308,"threads_started":1}
I20260806 09:02:17.920454 18097 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37657
I20260806 09:02:17.920779 18097 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:17.921523 18097 heartbeater.cc:511] Master 127.22.98.62:37657 requested a full tablet report, sending...
I20260806 09:02:17.923806 17819 ts_manager.cc:194] Registered new tserver with Master: 2ef72cab1fe84f70a85602c9a7402be1 (127.22.98.1:44981)
I20260806 09:02:17.925350 17819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.1@KRBTEST.COM'} at 127.22.98.1:45355
WARNING: no policy specified for kudu/127.22.98.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
Entry for principal kudu/127.22.98.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab.
I20260806 09:02:17.970674 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:18.324241 18107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:18.324563 18107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:18.324699 18107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:18.332407 18107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:18.332540 18107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:18.332756 18107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:18.343071 18107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.2.keytab
--principal=kudu/127.22.98.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:18.344528 18107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:18.345984 18107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:18.360217 18114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:18.360903 18113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:18.360932 18116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:18.361459 18107 server_base.cc:1034] running on GCE node
I20260806 09:02:18.362178 18107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:18.363343 18107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:18.364576 18107 hybrid_clock.cc:648] HybridClock initialized: now 1786006938364500 us; error 67 us; skew 500 ppm
I20260806 09:02:18.364934 18107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:18 dist-test-slave-jpj1 krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006938, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:18.368261 18107 init.cc:377] Logged in from keytab as kudu/127.22.98.2@KRBTEST.COM (short username kudu)
I20260806 09:02:18.369897 18107 webserver.cc:460] Webserver started at http://127.22.98.2:34499/ using document root <none> and password file <none>
I20260806 09:02:18.370623 18107 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:18.370736 18107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:18.371070 18107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:18.374236 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "db9cae49bc39433599e78d42b9b03365"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "08419f48eae010a1c042de15d8e506c9"
server_key_iv: "8de2a77ccae3753fcaa1c65e4aa86fc2"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.375052 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "db9cae49bc39433599e78d42b9b03365"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "08419f48eae010a1c042de15d8e506c9"
server_key_iv: "8de2a77ccae3753fcaa1c65e4aa86fc2"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.380584 18107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:02:18.384482 18127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:18.386030 18107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:18.386210 18107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "db9cae49bc39433599e78d42b9b03365"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "08419f48eae010a1c042de15d8e506c9"
server_key_iv: "8de2a77ccae3753fcaa1c65e4aa86fc2"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.386415 18107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:18.413199 18107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:18.417598 18107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:18.417906 18107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:18.418949 18107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:18.420444 18107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:18.420528 18107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:18.420647 18107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:18.420703 18107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:18.453212 18107 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:39149
I20260806 09:02:18.453226 18277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:39149 every 8 connection(s)
I20260806 09:02:18.454499 18107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:18.463704 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 18107
I20260806 09:02:18.463907 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:18.464349 22920 external_mini_cluster.cc:1428] Setting key 226bb562c0ca3a8bea68f43ff2cf2ce3
Aug 06 09:02:18 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006938, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.2@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:18.475382 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:18.457142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:60545 (local address 127.22.98.62:37657)
0806 09:02:18.457629 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:18.457676 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:18.458642 (+ 966us) server_negotiation.cc:407] Connection header received
0806 09:02:18.460612 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:18.460617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:18.460701 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:18.460805 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:18.462414 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:18.463185 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:18.465083 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:18.465585 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:18.465698 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:18.469402 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:18.469408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:18.469411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:18.469446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:18.472081 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:18.472898 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:18.472902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:18.472905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:18.473042 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:18.473519 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:18.473522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:18.473525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:18.473834 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:18.473971 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:18.474724 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 09:02:18.474986 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
I20260806 09:02:18.476397 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:18.457725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37657 (local address 127.22.98.2:60545)
0806 09:02:18.458462 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:18.458509 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:18.460305 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:18.461053 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:18.461062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:18.461608 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:18.462217 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:18.462236 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:18.463373 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:18.463379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:18.464928 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:18.464947 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:18.465817 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:18.465825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:18.466081 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:18.466789 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:18.466813 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:18.469169 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:18.472225 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:18.472235 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:18.472262 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:18.472770 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:18.473164 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:18.473167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:18.473170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:18.473402 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:18.473964 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:18.473973 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:18.474336 (+ 363us) client_negotiation.cc:769] Sending connection context
0806 09:02:18.474587 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 09:02:18.475151 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":276,"threads_started":1}
I20260806 09:02:18.477921 18280 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37657
I20260806 09:02:18.478241 18280 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:18.478993 18280 heartbeater.cc:511] Master 127.22.98.62:37657 requested a full tablet report, sending...
I20260806 09:02:18.480599 17819 ts_manager.cc:194] Registered new tserver with Master: db9cae49bc39433599e78d42b9b03365 (127.22.98.2:39149)
I20260806 09:02:18.481647 17819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.2@KRBTEST.COM'} at 127.22.98.2:60545
WARNING: no policy specified for kudu/127.22.98.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
Entry for principal kudu/127.22.98.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab.
I20260806 09:02:18.535288 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:18.880182 18287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:18.880471 18287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:18.880595 18287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:18.888047 18287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:18.888166 18287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:18.888314 18287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:18.900985 18287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.3.keytab
--principal=kudu/127.22.98.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:18.902318 18287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:18.903695 18287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:18.916914 18293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:18.916925 18295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:18.917038 18298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:18.917999 18287 server_base.cc:1034] running on GCE node
I20260806 09:02:18.918678 18287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:18.919893 18287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:18.921121 18287 hybrid_clock.cc:648] HybridClock initialized: now 1786006938921056 us; error 55 us; skew 500 ppm
I20260806 09:02:18.921509 18287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:18 dist-test-slave-jpj1 krb5kdc[17753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006938, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:18.924759 18287 init.cc:377] Logged in from keytab as kudu/127.22.98.3@KRBTEST.COM (short username kudu)
I20260806 09:02:18.926229 18287 webserver.cc:460] Webserver started at http://127.22.98.3:35069/ using document root <none> and password file <none>
I20260806 09:02:18.926921 18287 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:18.927026 18287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:18.927312 18287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:18.928553 18097 heartbeater.cc:503] Master 127.22.98.62:37657 was elected leader, sending a full tablet report...
I20260806 09:02:18.930718 18287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "20fd91f7fd884c548bb397f005fb667d"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "25c777185c398abe73a03c56e8056b6e"
server_key_iv: "761dbc877ea50a0f75c070ce2f970a04"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.931592 18287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "20fd91f7fd884c548bb397f005fb667d"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "25c777185c398abe73a03c56e8056b6e"
server_key_iv: "761dbc877ea50a0f75c070ce2f970a04"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.937089 18287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:02:18.940650 18307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:18.942056 18287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:02:18.942245 18287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "20fd91f7fd884c548bb397f005fb667d"
format_stamp: "Formatted at 2026-08-06 09:02:18 on dist-test-slave-jpj1"
server_key: "25c777185c398abe73a03c56e8056b6e"
server_key_iv: "761dbc877ea50a0f75c070ce2f970a04"
server_key_version: "encryptionkey@0"
I20260806 09:02:18.942438 18287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:18.972764 18287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:18.977249 18287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:18.977536 18287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:18.978530 18287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:18.979943 18287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:18.980021 18287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:18.980140 18287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:18.980207 18287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:19.011747 18287 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:42009
I20260806 09:02:19.011794 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:42009 every 8 connection(s)
I20260806 09:02:19.013046 18287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:19.017802 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 18287
I20260806 09:02:19.017983 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:19.018411 22920 external_mini_cluster.cc:1428] Setting key 0fed5d327613a094598a167cc22f4144
Aug 06 09:02:19 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006938, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.3@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:19.032578 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.015599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:52909 (local address 127.22.98.62:37657)
0806 09:02:19.016146 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.016153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.017116 (+ 963us) server_negotiation.cc:407] Connection header received
0806 09:02:19.019000 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.019006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.019088 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.019189 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:19.020750 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.021556 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.023077 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.023476 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.023610 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.026977 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:19.026982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:19.026985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:19.027018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:19.029479 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.030184 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.030189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.030191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.030324 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.030790 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.030795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.030797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.031072 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:19.031216 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.031966 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.032158 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":241,"threads_started":1}
I20260806 09:02:19.033643 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.016209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37657 (local address 127.22.98.3:52909)
0806 09:02:19.016951 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.016992 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.018713 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.019449 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.019458 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.020018 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:19.020583 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.020602 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.021707 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.021712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.022935 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.022949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.023714 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.023722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.023967 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.024556 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:02:19.024578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:19.026765 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:19.029622 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:19.029632 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:19.029650 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:19.030068 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:19.030431 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:02:19.030437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:02:19.030440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:02:19.030678 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:02:19.031202 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:19.031210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:19.031569 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.031860 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.032373 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":288,"threads_started":1}
I20260806 09:02:19.035022 18464 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37657
I20260806 09:02:19.035310 18464 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:19.036037 18464 heartbeater.cc:511] Master 127.22.98.62:37657 requested a full tablet report, sending...
I20260806 09:02:19.037400 17819 ts_manager.cc:194] Registered new tserver with Master: 20fd91f7fd884c548bb397f005fb667d (127.22.98.3:42009)
I20260806 09:02:19.038200 17819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.98.3@KRBTEST.COM'} at 127.22.98.3:52909
I20260806 09:02:19.045519 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:19.062098 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.049727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.22.98.62:37657)
0806 09:02:19.049942 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.049949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.050287 (+ 338us) server_negotiation.cc:407] Connection header received
0806 09:02:19.050450 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.050454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.050516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.050595 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:19.052006 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.052713 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.053924 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.054341 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.054431 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.056747 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:19.056752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:19.056756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:19.056787 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:19.059462 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.060167 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.060171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.060173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.060311 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.060718 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.060722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.060724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.061095 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:19.061289 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.061593 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.061767 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:02:19.066262 17819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:19.073410 17819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:19.112650 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.086952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:44981 (local address 127.0.0.1:57450)
0806 09:02:19.104158 (+ 17206us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.104179 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.104332 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.105328 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.105332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.105361 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.105727 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.105740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.107653 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.107658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.109931 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.109942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.111830 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.111838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.111999 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.112032 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.112135 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.112275 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17080,"thread_start_us":16117,"threads_started":1}
I20260806 09:02:19.113179 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.086641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.22.98.2:39149)
0806 09:02:19.103637 (+ 16996us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.103646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.103676 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:02:19.103819 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.103830 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.104130 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.105275 (+ 1145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.105540 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.107137 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.109807 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.111246 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.112426 (+ 1180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.112618 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.112692 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.112864 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16798,"thread_start_us":17679,"threads_started":1}
I20260806 09:02:19.113229 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.087365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.22.98.3:42009)
0806 09:02:19.103017 (+ 15652us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.103028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.103598 (+ 570us) server_negotiation.cc:407] Connection header received
0806 09:02:19.103835 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.103844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.104135 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.104382 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.105811 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.106854 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.109526 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.111018 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.111837 (+ 819us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.112061 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.112717 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.112887 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15439,"thread_start_us":15233,"threads_started":1}
I20260806 09:02:19.113562 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.086757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.22.98.1:44981)
0806 09:02:19.100174 (+ 13417us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.100185 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.104476 (+ 4291us) server_negotiation.cc:407] Connection header received
0806 09:02:19.104603 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.104611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.104837 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.105051 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.105903 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.107441 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.110094 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.111292 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.111418 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.111679 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.113069 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.113280 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13188,"thread_start_us":12994,"threads_started":1}
I20260806 09:02:19.113150 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.086729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:42009 (local address 127.0.0.1:46478)
0806 09:02:19.103396 (+ 16667us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.103421 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.103639 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.104688 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.104692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.104717 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.105459 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.105474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.107003 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.107008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.109360 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.109373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.112208 (+ 2835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.112214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.112496 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.112527 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.112629 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.112769 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16520,"mutex_wait_us":1,"thread_start_us":9511,"threads_started":1}
I20260806 09:02:19.113698 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.086389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39149 (local address 127.0.0.1:45210)
0806 09:02:19.096682 (+ 10293us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.096727 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.096967 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.104637 (+ 7670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.104645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.104677 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.105198 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.105212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.107294 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.107300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.109638 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.109653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.111491 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.111499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.111736 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.111807 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.111931 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.112147 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10004,"mutex_wait_us":5,"thread_start_us":17507,"threads_started":1}
I20260806 09:02:19.117456 18377 tablet_service.cc:1467] Processing CreateTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 (DEFAULT_TABLE table=test-table [id=470464b43d4a4209a6a7d278474de9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:19.117455 18190 tablet_service.cc:1467] Processing CreateTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 (DEFAULT_TABLE table=test-table [id=470464b43d4a4209a6a7d278474de9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:19.118692 18190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94a92d6a78450083d83d0ad4faaeb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:19.118695 18377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94a92d6a78450083d83d0ad4faaeb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:19.122593 18001 tablet_service.cc:1467] Processing CreateTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 (DEFAULT_TABLE table=test-table [id=470464b43d4a4209a6a7d278474de9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:19.124266 18001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94a92d6a78450083d83d0ad4faaeb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:19.130568 18492 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Bootstrap starting.
I20260806 09:02:19.133602 18493 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Bootstrap starting.
I20260806 09:02:19.135807 18492 tablet_bootstrap.cc:654] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:19.137247 18492 log.cc:824] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:19.137477 18493 tablet_bootstrap.cc:654] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:19.138825 18493 log.cc:824] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:19.139559 18495 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Bootstrap starting.
I20260806 09:02:19.140969 18492 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: No bootstrap required, opened a new log
I20260806 09:02:19.141306 18492 ts_tablet_manager.cc:1397] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 09:02:19.142756 18495 tablet_bootstrap.cc:654] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:19.144197 18495 log.cc:824] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:19.145888 18493 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: No bootstrap required, opened a new log
I20260806 09:02:19.146222 18493 ts_tablet_manager.cc:1397] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260806 09:02:19.147722 18495 tablet_bootstrap.cc:492] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: No bootstrap required, opened a new log
I20260806 09:02:19.148003 18495 ts_tablet_manager.cc:1397] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:02:19.150645 18492 raft_consensus.cc:348] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.151031 18492 raft_consensus.cc:374] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:19.151168 18492 raft_consensus.cc:729] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20fd91f7fd884c548bb397f005fb667d, State: Initialized, Role: FOLLOWER
I20260806 09:02:19.151803 18492 consensus_queue.cc:260] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.152963 18464 heartbeater.cc:503] Master 127.22.98.62:37657 was elected leader, sending a full tablet report...
I20260806 09:02:19.153439 18492 ts_tablet_manager.cc:1428] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260806 09:02:19.156867 18493 raft_consensus.cc:348] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.157334 18493 raft_consensus.cc:374] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:19.157441 18493 raft_consensus.cc:729] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db9cae49bc39433599e78d42b9b03365, State: Initialized, Role: FOLLOWER
I20260806 09:02:19.158097 18493 consensus_queue.cc:260] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.159363 18280 heartbeater.cc:503] Master 127.22.98.62:37657 was elected leader, sending a full tablet report...
I20260806 09:02:19.159315 18495 raft_consensus.cc:348] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.159798 18495 raft_consensus.cc:374] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:19.159905 18495 raft_consensus.cc:729] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef72cab1fe84f70a85602c9a7402be1, State: Initialized, Role: FOLLOWER
I20260806 09:02:19.160156 18493 ts_tablet_manager.cc:1428] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260806 09:02:19.160535 18495 consensus_queue.cc:260] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.162091 18495 ts_tablet_manager.cc:1428] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260806 09:02:19.206553 18281 tablet.cc:2369] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:19.264719 18465 tablet.cc:2369] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:19.398082 18099 tablet.cc:2369] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:19.500825 18500 raft_consensus.cc:482] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:19.501183 18500 raft_consensus.cc:504] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.502990 18500 leader_election.cc:288] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ef72cab1fe84f70a85602c9a7402be1 (127.22.98.1:44981), 20fd91f7fd884c548bb397f005fb667d (127.22.98.3:42009)
I20260806 09:02:19.509966 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.503507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:36121 (local address 127.22.98.3:42009)
0806 09:02:19.503741 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.503747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.504600 (+ 853us) server_negotiation.cc:407] Connection header received
0806 09:02:19.504776 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.504781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.504844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.504919 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.505591 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.506301 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.508244 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.509028 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.509162 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.509272 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.509566 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.509709 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 09:02:19.510078 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.503413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:51749 (local address 127.22.98.1:44981)
0806 09:02:19.503657 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.503664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.504532 (+ 868us) server_negotiation.cc:407] Connection header received
0806 09:02:19.504732 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.504738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.504810 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.504889 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.505499 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.506228 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.508290 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.509115 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.509248 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.509357 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.509700 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.509834 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:02:19.510159 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.503476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:42009 (local address 127.22.98.2:36121)
0806 09:02:19.504492 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.504507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.504645 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.504943 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.504947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.504969 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.505418 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.505429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.506424 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.506432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.508077 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.508093 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.509252 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.509259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.509404 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.509433 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.509519 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.509684 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":22,"thread_start_us":479,"threads_started":1}
I20260806 09:02:19.510238 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.503434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:44981 (local address 127.22.98.2:51749)
0806 09:02:19.504409 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.504426 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.504589 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.504924 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.504928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.504952 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.505343 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.505354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.506377 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.506383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.508112 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.508128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.509358 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.509364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.509510 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.509556 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.509672 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.509806 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":24,"thread_start_us":541,"threads_started":1}
I20260806 09:02:19.510735 18501 raft_consensus.cc:482] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:19.511083 18501 raft_consensus.cc:504] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.511404 18030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "db9cae49bc39433599e78d42b9b03365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef72cab1fe84f70a85602c9a7402be1" is_pre_election: true
I20260806 09:02:19.511476 18402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "db9cae49bc39433599e78d42b9b03365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20fd91f7fd884c548bb397f005fb667d" is_pre_election: true
I20260806 09:02:19.512009 18402 raft_consensus.cc:2454] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9cae49bc39433599e78d42b9b03365 in term 0.
I20260806 09:02:19.512096 18030 raft_consensus.cc:2454] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9cae49bc39433599e78d42b9b03365 in term 0.
I20260806 09:02:19.512809 18130 leader_election.cc:302] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20fd91f7fd884c548bb397f005fb667d, db9cae49bc39433599e78d42b9b03365; no voters:
I20260806 09:02:19.513330 18501 leader_election.cc:288] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db9cae49bc39433599e78d42b9b03365 (127.22.98.2:39149), 20fd91f7fd884c548bb397f005fb667d (127.22.98.3:42009)
I20260806 09:02:19.513562 18500 raft_consensus.cc:2781] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:19.513708 18500 raft_consensus.cc:482] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:19.513803 18500 raft_consensus.cc:3037] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:19.516992 18500 raft_consensus.cc:504] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.517868 18500 leader_election.cc:288] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [CANDIDATE]: Term 1 election: Requested vote from peers 2ef72cab1fe84f70a85602c9a7402be1 (127.22.98.1:44981), 20fd91f7fd884c548bb397f005fb667d (127.22.98.3:42009)
I20260806 09:02:19.518412 18030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "db9cae49bc39433599e78d42b9b03365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef72cab1fe84f70a85602c9a7402be1"
I20260806 09:02:19.518527 18402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "db9cae49bc39433599e78d42b9b03365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20fd91f7fd884c548bb397f005fb667d"
I20260806 09:02:19.518679 18030 raft_consensus.cc:3037] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:19.518723 18402 raft_consensus.cc:3037] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:19.522097 18030 raft_consensus.cc:2454] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db9cae49bc39433599e78d42b9b03365 in term 1.
I20260806 09:02:19.522140 18402 raft_consensus.cc:2454] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db9cae49bc39433599e78d42b9b03365 in term 1.
I20260806 09:02:19.522387 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.514267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:57713 (local address 127.22.98.3:42009)
0806 09:02:19.514434 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.514440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.515285 (+ 845us) server_negotiation.cc:407] Connection header received
0806 09:02:19.515470 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.515475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.515572 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.515634 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.516381 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.517402 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.519910 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.521208 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.521306 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.521419 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.521969 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.522104 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 09:02:19.522457 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.513540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:39149 (local address 127.22.98.1:46809)
0806 09:02:19.514796 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.514819 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.515026 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.515387 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.515392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.515420 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.515815 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.515828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.516954 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.516960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.519206 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.519223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.521038 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.521044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.521221 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.521256 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.521931 (+ 675us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.522075 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":1317,"threads_started":1}
I20260806 09:02:19.521865 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.513916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:46809 (local address 127.22.98.2:39149)
0806 09:02:19.514091 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.514097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.514901 (+ 804us) server_negotiation.cc:407] Connection header received
0806 09:02:19.515189 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.515194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.515283 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.515376 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:19.515969 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.516835 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.519397 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.520610 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.520729 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.520827 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.521419 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.521571 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 09:02:19.522755 18130 leader_election.cc:302] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20fd91f7fd884c548bb397f005fb667d, db9cae49bc39433599e78d42b9b03365; no voters:
I20260806 09:02:19.523084 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.513648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:42009 (local address 127.22.98.1:57713)
0806 09:02:19.515172 (+ 1524us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:19.515190 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:19.515316 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:19.515831 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:19.515835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:19.515858 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:19.516217 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.516230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.517556 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.517562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.519715 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:19.519733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.521608 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.521614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:19.521782 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.521812 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 09:02:19.522564 (+ 752us) client_negotiation.cc:240] Negotiation successful
0806 09:02:19.522726 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"mutex_wait_us":1,"thread_start_us":752,"threads_started":1}
I20260806 09:02:19.523320 18500 raft_consensus.cc:2781] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:19.523821 18402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "2ef72cab1fe84f70a85602c9a7402be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20fd91f7fd884c548bb397f005fb667d" is_pre_election: true
I20260806 09:02:19.524113 18402 raft_consensus.cc:2379] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ef72cab1fe84f70a85602c9a7402be1 in current term 1: Already voted for candidate db9cae49bc39433599e78d42b9b03365 in this term.
I20260806 09:02:19.523854 18216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94a92d6a78450083d83d0ad4faaeb5" candidate_uuid: "2ef72cab1fe84f70a85602c9a7402be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9cae49bc39433599e78d42b9b03365" is_pre_election: true
I20260806 09:02:19.527865 18500 raft_consensus.cc:686] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 1 LEADER]: Becoming Leader. State: Replica: db9cae49bc39433599e78d42b9b03365, State: Running, Role: LEADER
I20260806 09:02:19.528422 18500 consensus_queue.cc:237] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } }
I20260806 09:02:19.532923 17935 leader_election.cc:302] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ef72cab1fe84f70a85602c9a7402be1; no voters: 20fd91f7fd884c548bb397f005fb667d, db9cae49bc39433599e78d42b9b03365
I20260806 09:02:19.533452 18501 raft_consensus.cc:2726] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 09:02:19.534217 17818 catalog_manager.cc:5495] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 reported cstate change: term changed from 0 to 1, leader changed from <none> to db9cae49bc39433599e78d42b9b03365 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "db9cae49bc39433599e78d42b9b03365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9cae49bc39433599e78d42b9b03365" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 39149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 } health_report { overall_health: UNKNOWN } } }
I20260806 09:02:19.573324 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.567134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.22.98.2:39149)
0806 09:02:19.567298 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.567302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.567656 (+ 354us) server_negotiation.cc:407] Connection header received
0806 09:02:19.567858 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.567861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.567969 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.568131 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:19.568479 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.569335 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.570727 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.571306 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.571436 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.572099 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:19.572720 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:19.572879 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.572972 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.573051 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:02:19.582903 18402 raft_consensus.cc:1261] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 1 FOLLOWER]: Refusing update from remote peer db9cae49bc39433599e78d42b9b03365: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:19.582947 18030 raft_consensus.cc:1261] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 1 FOLLOWER]: Refusing update from remote peer db9cae49bc39433599e78d42b9b03365: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:19.583986 18508 consensus_queue.cc:1035] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20fd91f7fd884c548bb397f005fb667d" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:19.584360 18500 consensus_queue.cc:1035] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef72cab1fe84f70a85602c9a7402be1" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 44981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:19.597343 18517 mvcc.cc:205] Tried to move back new op lower bound from 7315484424520577024 to 7315484424312295424. Current Snapshot: MvccSnapshot[applied={T|T < 7315484424520577024}]
I20260806 09:02:19.609122 17818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:19.609469 17818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:19.613523 17818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 181a117204fb4d06bcf84de160795e46: Sending DeleteTablet for 3 replicas of tablet af94a92d6a78450083d83d0ad4faaeb5
I20260806 09:02:19.614871 18190 tablet_service.cc:1514] Processing DeleteTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:19 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:45210
I20260806 09:02:19.615067 18377 tablet_service.cc:1514] Processing DeleteTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:19 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:46478
I20260806 09:02:19.615195 18001 tablet_service.cc:1514] Processing DeleteTablet for tablet af94a92d6a78450083d83d0ad4faaeb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:19 UTC) from {username='kudu', principal='kudu/127.22.98.62@KRBTEST.COM'} at 127.0.0.1:57450
I20260806 09:02:19.620390 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 17907
I20260806 09:02:19.627893 18529 tablet_replica.cc:330] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: stopping tablet replica
I20260806 09:02:19.628121 18531 tablet_replica.cc:330] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: stopping tablet replica
I20260806 09:02:19.628461 18529 raft_consensus.cc:2229] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:19.628628 18531 raft_consensus.cc:2229] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:19.629117 18529 raft_consensus.cc:2258] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:19.629238 18531 raft_consensus.cc:2258] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:19.629321 18528 tablet_replica.cc:330] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: stopping tablet replica
I20260806 09:02:19.629791 18528 raft_consensus.cc:2229] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:19.630529 18528 raft_consensus.cc:2258] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:19.632539 18531 ts_tablet_manager.cc:1905] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:19.632532 18528 ts_tablet_manager.cc:1905] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:19.632530 18529 ts_tablet_manager.cc:1905] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:19.639582 18529 ts_tablet_manager.cc:1918] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:19.639873 18529 log.cc:1196] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/af94a92d6a78450083d83d0ad4faaeb5
I20260806 09:02:19.640021 18528 ts_tablet_manager.cc:1918] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:19.640229 18528 log.cc:1196] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/af94a92d6a78450083d83d0ad4faaeb5
Aug 06 09:02:19 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.1@KRBTEST.COM
I20260806 09:02:19.640619 18529 ts_tablet_manager.cc:1939] T af94a92d6a78450083d83d0ad4faaeb5 P 20fd91f7fd884c548bb397f005fb667d: Deleting consensus metadata
I20260806 09:02:19.640787 18528 ts_tablet_manager.cc:1939] T af94a92d6a78450083d83d0ad4faaeb5 P db9cae49bc39433599e78d42b9b03365: Deleting consensus metadata
I20260806 09:02:19.642503 18531 ts_tablet_manager.cc:1918] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:19.642740 18531 log.cc:1196] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/af94a92d6a78450083d83d0ad4faaeb5
I20260806 09:02:19.642681 17800 catalog_manager.cc:4841] TS 20fd91f7fd884c548bb397f005fb667d (127.22.98.3:42009): tablet af94a92d6a78450083d83d0ad4faaeb5 (table test-table [id=470464b43d4a4209a6a7d278474de9e4]) successfully deleted
I20260806 09:02:19.643045 17801 catalog_manager.cc:4841] TS db9cae49bc39433599e78d42b9b03365 (127.22.98.2:39149): tablet af94a92d6a78450083d83d0ad4faaeb5 (table test-table [id=470464b43d4a4209a6a7d278474de9e4]) successfully deleted
I20260806 09:02:19.643330 18531 ts_tablet_manager.cc:1939] T af94a92d6a78450083d83d0ad4faaeb5 P 2ef72cab1fe84f70a85602c9a7402be1: Deleting consensus metadata
I20260806 09:02:19.645406 17797 catalog_manager.cc:4841] TS 2ef72cab1fe84f70a85602c9a7402be1 (127.22.98.1:44981): tablet af94a92d6a78450083d83d0ad4faaeb5 (table test-table [id=470464b43d4a4209a6a7d278474de9e4]) successfully deleted
I20260806 09:02:19.648880 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.622058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.22.98.1:44981)
0806 09:02:19.622256 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.622264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.628661 (+ 6397us) server_negotiation.cc:407] Connection header received
0806 09:02:19.628869 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.628875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.628963 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.629081 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:19.631341 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.632450 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.634265 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.634889 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.634986 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.641963 (+ 6977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:19.641980 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:19.641999 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:19.642053 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:19.645765 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.646559 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.646569 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.646572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.646736 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.647239 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.647243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.647245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.647734 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:19.647988 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.648405 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.648596 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:02:19.649614 18088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:19.743821 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 17907
I20260806 09:02:19.770939 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 18107
Aug 06 09:02:19 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.2@KRBTEST.COM
I20260806 09:02:19.790088 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.771866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.22.98.2:39149)
0806 09:02:19.772059 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.772067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.772355 (+ 288us) server_negotiation.cc:407] Connection header received
0806 09:02:19.772538 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.772542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.772628 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.772713 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:19.774506 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.775313 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.777182 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.777637 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.777735 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.783424 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:19.783454 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:19.783473 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:19.783515 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:19.786821 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.787794 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.787802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.787805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.787973 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.788397 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.788401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.788403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.788867 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:19.789109 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.789488 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.789761 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 09:02:19.790987 18272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:19.907941 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 18107
I20260806 09:02:19.929471 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 18287
Aug 06 09:02:19 dist-test-slave-jpj1 krb5kdc[17753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.3@KRBTEST.COM
I20260806 09:02:19.945119 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:19.930284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.22.98.3:42009)
0806 09:02:19.930496 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:19.930501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:19.930524 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:02:19.930703 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:19.930707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:19.930799 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:19.930890 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:19.932281 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.932998 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.934309 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:19.934729 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:19.934853 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:19.939256 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:19.939273 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:19.939292 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:19.939329 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:19.942316 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.943011 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.943020 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.943022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.943162 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:19.943789 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:19.943793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:19.943795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:19.944144 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:19.944395 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:19.944672 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 09:02:19.944879 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:02:19.945830 18459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:20.034917 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 18287
I20260806 09:02:20.056512 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 17771
I20260806 09:02:20.057685 17861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:20.169351 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 17771
2026-08-06T09:02:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4953 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T09:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:20Z Disabled control of system clock
I20260806 09:02:20.268193 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37257
--webserver_interface=127.22.98.62
--webserver_port=0
--builtin_ntp_servers=127.22.98.20:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:20.630925 18544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:20.631254 18544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:20.631345 18544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:20.638908 18544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:20.639053 18544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:20.639155 18544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:20.639226 18544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:20.639273 18544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:20.649104 18544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37257
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:20.650554 18544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:20.652452 18544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:20.664969 18551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:20.664969 18550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:20.664999 18553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:20.665985 18544 server_base.cc:1034] running on GCE node
I20260806 09:02:20.666916 18544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:20.668526 18544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:20.669766 18544 hybrid_clock.cc:648] HybridClock initialized: now 1786006940669709 us; error 42 us; skew 500 ppm
I20260806 09:02:20.670169 18544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:20.672732 18544 webserver.cc:460] Webserver started at http://127.22.98.62:46739/ using document root <none> and password file <none>
I20260806 09:02:20.673496 18544 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:20.673619 18544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:20.673943 18544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:20.677064 18544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "152321f845474b1e9a4650cb4c19a90a"
format_stamp: "Formatted at 2026-08-06 09:02:20 on dist-test-slave-jpj1"
server_key: "345ddc9e6f65ea527539e810378b8950"
server_key_iv: "b603c604b0aa1987b11a1f725cba1f27"
server_key_version: "encryptionkey@0"
I20260806 09:02:20.677938 18544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "152321f845474b1e9a4650cb4c19a90a"
format_stamp: "Formatted at 2026-08-06 09:02:20 on dist-test-slave-jpj1"
server_key: "345ddc9e6f65ea527539e810378b8950"
server_key_iv: "b603c604b0aa1987b11a1f725cba1f27"
server_key_version: "encryptionkey@0"
I20260806 09:02:20.683759 18544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:02:20.687690 18561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:20.689137 18544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:20.689350 18544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "152321f845474b1e9a4650cb4c19a90a"
format_stamp: "Formatted at 2026-08-06 09:02:20 on dist-test-slave-jpj1"
server_key: "345ddc9e6f65ea527539e810378b8950"
server_key_iv: "b603c604b0aa1987b11a1f725cba1f27"
server_key_version: "encryptionkey@0"
I20260806 09:02:20.689595 18544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:20.707623 18544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:20.708575 18544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:20.708869 18544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:20.728096 18544 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37257
I20260806 09:02:20.728108 18627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37257 every 8 connection(s)
I20260806 09:02:20.729535 18544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:20.730908 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 18544
I20260806 09:02:20.731096 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:20.731701 22920 external_mini_cluster.cc:1428] Setting key 1e77f6b4454fc0785f13c23a1da1a37a
I20260806 09:02:20.734697 18628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:20.747360 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:20.733861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.22.98.62:37257)
0806 09:02:20.735250 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:20.735263 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:20.735316 (+ 53us) server_negotiation.cc:407] Connection header received
0806 09:02:20.737305 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:20.737331 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:20.737871 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:20.738289 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:20.738954 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:20.740520 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:20.742539 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:20.743253 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:20.743425 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:20.744760 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:20.744773 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:20.744796 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:20.745435 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:20.745518 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:20.745542 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:20.745642 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:20.745895 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 09:02:20.746268 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":465,"threads_started":1}
I20260806 09:02:20.748466 18628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: Bootstrap starting.
I20260806 09:02:20.751982 18628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:20.752985 18628 log.cc:824] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:20.756230 18628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: No bootstrap required, opened a new log
I20260806 09:02:20.764158 18628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } }
I20260806 09:02:20.764564 18628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:20.764698 18628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152321f845474b1e9a4650cb4c19a90a, State: Initialized, Role: FOLLOWER
I20260806 09:02:20.765259 18628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } }
I20260806 09:02:20.765467 18628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:20.765574 18628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:20.765722 18628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:20.768003 18628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } }
I20260806 09:02:20.768541 18628 leader_election.cc:302] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 152321f845474b1e9a4650cb4c19a90a; no voters:
I20260806 09:02:20.769141 18628 leader_election.cc:288] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:20.769331 18634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:20.770601 18634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [term 1 LEADER]: Becoming Leader. State: Replica: 152321f845474b1e9a4650cb4c19a90a, State: Running, Role: LEADER
I20260806 09:02:20.771122 18634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } }
I20260806 09:02:20.771678 18628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:20.774056 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "152321f845474b1e9a4650cb4c19a90a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } } }
I20260806 09:02:20.774221 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 152321f845474b1e9a4650cb4c19a90a. Latest consensus state: current_term: 1 leader_uuid: "152321f845474b1e9a4650cb4c19a90a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152321f845474b1e9a4650cb4c19a90a" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37257 } } }
I20260806 09:02:20.774497 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:20.774494 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:20.775990 18647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:20.781886 18647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:20.792191 18647 catalog_manager.cc:1347] Generated new cluster ID: d4a0390962844416a87ef5f6aaa450ab
I20260806 09:02:20.792341 18647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:20.811506 18647 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:20.812916 18647 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:20.826905 18647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: Generated new TSK 0
I20260806 09:02:20.827847 18647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:20.835119 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.1:0
--local_ip_for_outbound_sockets=127.22.98.1
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--builtin_ntp_servers=127.22.98.20:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:02:21.212760 18665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:21.213142 18665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:21.213238 18665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:21.220753 18665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:21.220877 18665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:21.220988 18665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.1
I20260806 09:02:21.231814 18665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.98.1
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.1
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:21.233227 18665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:21.235435 18665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:21.250963 18671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:21.251066 18675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:21.250970 18672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:21.251778 18665 server_base.cc:1034] running on GCE node
I20260806 09:02:21.252533 18665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:21.253659 18665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:21.254979 18665 hybrid_clock.cc:648] HybridClock initialized: now 1786006941254917 us; error 45 us; skew 500 ppm
I20260806 09:02:21.255399 18665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:21.258163 18665 webserver.cc:460] Webserver started at http://127.22.98.1:41253/ using document root <none> and password file <none>
I20260806 09:02:21.258960 18665 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:21.259063 18665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:21.259379 18665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:21.262738 18665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data/instance:
uuid: "b6ba0512970c4dfb8b236142c8684ed3"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "8145eb21968653cc5550924b1986cfbf"
server_key_iv: "fffb63e2951c196b220b298a855b07f9"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.263692 18665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance:
uuid: "b6ba0512970c4dfb8b236142c8684ed3"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "8145eb21968653cc5550924b1986cfbf"
server_key_iv: "fffb63e2951c196b220b298a855b07f9"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.269598 18665 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 09:02:21.273663 18684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.275110 18665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:02:21.275292 18665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal
uuid: "b6ba0512970c4dfb8b236142c8684ed3"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "8145eb21968653cc5550924b1986cfbf"
server_key_iv: "fffb63e2951c196b220b298a855b07f9"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.275569 18665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:21.303020 18665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:21.303954 18665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:21.304208 18665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:21.305212 18665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:21.306861 18665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:21.306954 18665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.307044 18665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:21.307099 18665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.339244 18665 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.1:41049
I20260806 09:02:21.339330 18837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.1:41049 every 8 connection(s)
I20260806 09:02:21.340709 18665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/data/info.pb
I20260806 09:02:21.347066 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 18665
I20260806 09:02:21.347214 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal/instance
I20260806 09:02:21.347739 22920 external_mini_cluster.cc:1428] Setting key ab6fc10bbcac79e67f7ab86133ace595
I20260806 09:02:21.351063 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.2:0
--local_ip_for_outbound_sockets=127.22.98.2
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--builtin_ntp_servers=127.22.98.20:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:21.355520 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:21.343262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.1:56629 (local address 127.22.98.62:37257)
0806 09:02:21.343710 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:21.343715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:21.345105 (+ 1390us) server_negotiation.cc:407] Connection header received
0806 09:02:21.347041 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:21.347049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:21.347151 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:21.347273 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:21.348648 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.349658 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.352189 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.352855 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.352962 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:21.354672 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:21.354678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:21.354681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:21.354702 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:21.354750 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:21.354797 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:21.354871 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:21.355115 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 09:02:21.355207 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":213,"threads_started":1}
I20260806 09:02:21.356942 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:21.343841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37257 (local address 127.22.98.1:56629)
0806 09:02:21.344911 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:21.344953 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:21.346703 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:21.347597 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:21.347608 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:21.347713 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:21.348442 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:21.348470 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.349841 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.349848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:21.351934 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:21.351952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.353121 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.353128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:21.353442 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:21.354230 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:21.354233 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:21.354311 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:21.354425 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:21.354894 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:21.354906 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:21.354914 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 09:02:21.355101 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 09:02:21.355520 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260806 09:02:21.358798 18838 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37257
I20260806 09:02:21.359134 18838 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:21.360057 18838 heartbeater.cc:511] Master 127.22.98.62:37257 requested a full tablet report, sending...
I20260806 09:02:21.362447 18582 ts_manager.cc:194] Registered new tserver with Master: b6ba0512970c4dfb8b236142c8684ed3 (127.22.98.1:41049)
I20260806 09:02:21.364068 18582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.1:56629
W20260806 09:02:21.707692 18842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:21.708011 18842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:21.708142 18842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:21.715525 18842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:21.715709 18842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:21.715862 18842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.2
I20260806 09:02:21.725034 18842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.98.2
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.2
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:21.726336 18842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:21.728276 18842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:21.741854 18851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:21.741897 18852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:21.742837 18854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:21.744577 18842 server_base.cc:1034] running on GCE node
I20260806 09:02:21.745265 18842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:21.746335 18842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:21.747582 18842 hybrid_clock.cc:648] HybridClock initialized: now 1786006941747498 us; error 57 us; skew 500 ppm
I20260806 09:02:21.747936 18842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:21.750352 18842 webserver.cc:460] Webserver started at http://127.22.98.2:35943/ using document root <none> and password file <none>
I20260806 09:02:21.751070 18842 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:21.751183 18842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:21.751515 18842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:21.754583 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data/instance:
uuid: "62a2031c8ce64ab2a7521af7648c7101"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "7b03a49f74201cc57b0cd5abc44deb26"
server_key_iv: "7f2f5a8d1249754be51cf6057678aebb"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.755391 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance:
uuid: "62a2031c8ce64ab2a7521af7648c7101"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "7b03a49f74201cc57b0cd5abc44deb26"
server_key_iv: "7f2f5a8d1249754be51cf6057678aebb"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.760856 18842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.002s
I20260806 09:02:21.764533 18860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.765959 18842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:02:21.766165 18842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal
uuid: "62a2031c8ce64ab2a7521af7648c7101"
format_stamp: "Formatted at 2026-08-06 09:02:21 on dist-test-slave-jpj1"
server_key: "7b03a49f74201cc57b0cd5abc44deb26"
server_key_iv: "7f2f5a8d1249754be51cf6057678aebb"
server_key_version: "encryptionkey@0"
I20260806 09:02:21.766392 18842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:21.798621 18842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:21.799706 18842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:21.799985 18842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:21.801080 18842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:21.802702 18842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:21.802809 18842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.802914 18842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:21.802989 18842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:21.833423 18842 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.2:34405
I20260806 09:02:21.833495 19014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.2:34405 every 8 connection(s)
I20260806 09:02:21.834784 18842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/data/info.pb
I20260806 09:02:21.835749 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 18842
I20260806 09:02:21.835912 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal/instance
I20260806 09:02:21.836351 22920 external_mini_cluster.cc:1428] Setting key 51298eb55e0a36ef5126ff81ee67c10c
I20260806 09:02:21.839506 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.98.3:0
--local_ip_for_outbound_sockets=127.22.98.3
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--builtin_ntp_servers=127.22.98.20:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:02:21.847702 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:21.837491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:59775 (local address 127.22.98.62:37257)
0806 09:02:21.837678 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:21.837684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:21.839184 (+ 1500us) server_negotiation.cc:407] Connection header received
0806 09:02:21.841042 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:21.841048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:21.841139 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:21.841236 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:21.842469 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.843202 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.844877 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.845301 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.845418 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:21.846830 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:21.846836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:21.846839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:21.846859 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:21.846896 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:21.846909 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:21.846986 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:21.847226 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 09:02:21.847338 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:02:21.848861 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:21.838032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37257 (local address 127.22.98.2:59775)
0806 09:02:21.839027 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:21.839073 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:21.840770 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:21.841581 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:21.841589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:21.841682 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:21.842288 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:21.842306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.843365 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.843371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:21.844702 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:21.844717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:21.845538 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:21.845546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:21.845773 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:21.846491 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:21.846494 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:21.846537 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:21.846624 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:21.847004 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:21.847016 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:21.847024 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 09:02:21.847198 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 09:02:21.847686 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":308,"threads_started":1}
I20260806 09:02:21.850446 19015 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37257
I20260806 09:02:21.850745 19015 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:21.851610 19015 heartbeater.cc:511] Master 127.22.98.62:37257 requested a full tablet report, sending...
I20260806 09:02:21.853263 18582 ts_manager.cc:194] Registered new tserver with Master: 62a2031c8ce64ab2a7521af7648c7101 (127.22.98.2:34405)
I20260806 09:02:21.854089 18582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.2:59775
W20260806 09:02:22.189467 19018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:22.189896 19018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:22.190025 19018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:22.197594 19018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:22.197723 19018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:22.197870 19018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.98.3
I20260806 09:02:22.207214 19018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.98.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.98.3
--webserver_port=0
--tserver_master_addrs=127.22.98.62:37257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.98.3
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:22.208671 19018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:22.210561 19018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:22.224922 19026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:22.224967 19029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:22.225005 19025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:22.225648 19018 server_base.cc:1034] running on GCE node
I20260806 09:02:22.226346 19018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:22.227351 19018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:22.228598 19018 hybrid_clock.cc:648] HybridClock initialized: now 1786006942228535 us; error 64 us; skew 500 ppm
I20260806 09:02:22.228966 19018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:02:22.231388 19018 webserver.cc:460] Webserver started at http://127.22.98.3:42709/ using document root <none> and password file <none>
I20260806 09:02:22.232196 19018 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:22.232290 19018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:22.232575 19018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:22.235642 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data/instance:
uuid: "573eb92dab374c5f814fd7d3b5d3c6a0"
format_stamp: "Formatted at 2026-08-06 09:02:22 on dist-test-slave-jpj1"
server_key: "e35dbe8c4cc34cf9792c626f741397de"
server_key_iv: "287ac355fe32a3dc76d402496a08e90a"
server_key_version: "encryptionkey@0"
I20260806 09:02:22.236487 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance:
uuid: "573eb92dab374c5f814fd7d3b5d3c6a0"
format_stamp: "Formatted at 2026-08-06 09:02:22 on dist-test-slave-jpj1"
server_key: "e35dbe8c4cc34cf9792c626f741397de"
server_key_iv: "287ac355fe32a3dc76d402496a08e90a"
server_key_version: "encryptionkey@0"
I20260806 09:02:22.241968 19018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:02:22.245754 19039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:22.247309 19018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:02:22.247488 19018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal
uuid: "573eb92dab374c5f814fd7d3b5d3c6a0"
format_stamp: "Formatted at 2026-08-06 09:02:22 on dist-test-slave-jpj1"
server_key: "e35dbe8c4cc34cf9792c626f741397de"
server_key_iv: "287ac355fe32a3dc76d402496a08e90a"
server_key_version: "encryptionkey@0"
I20260806 09:02:22.247789 19018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:22.268288 19018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:22.269169 19018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:22.269443 19018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:22.270542 19018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:02:22.272270 19018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:02:22.272360 19018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:22.272468 19018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:02:22.272535 19018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:22.304010 19018 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.3:38679
I20260806 09:02:22.304091 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.3:38679 every 8 connection(s)
I20260806 09:02:22.305387 19018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/data/info.pb
I20260806 09:02:22.313771 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19018
I20260806 09:02:22.313951 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal/instance
I20260806 09:02:22.314565 22920 external_mini_cluster.cc:1428] Setting key c97794a666e966d3530648455e39bdf4
I20260806 09:02:22.319084 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.308095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.3:45817 (local address 127.22.98.62:37257)
0806 09:02:22.308280 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.308286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.309868 (+ 1582us) server_negotiation.cc:407] Connection header received
0806 09:02:22.311667 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.311677 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.311751 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.311842 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:22.313037 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.313866 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.316070 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.316608 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.316728 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.318280 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:22.318286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:22.318289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:22.318308 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:22.318343 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:22.318356 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:22.318430 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.318722 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.318818 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:02:22.320178 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.308599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37257 (local address 127.22.98.3:45817)
0806 09:02:22.309648 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.309739 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.311384 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.312095 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.312104 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.312197 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:02:22.312823 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.312843 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.314044 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.314049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.315895 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.315914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.316855 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.316861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.317075 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.317939 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:02:22.317942 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:02:22.317985 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:02:22.318078 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:02:22.318478 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:02:22.318489 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:02:22.318496 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.318717 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.319051 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":350,"threads_started":1}
I20260806 09:02:22.321548 19186 heartbeater.cc:347] Connected to a master server at 127.22.98.62:37257
I20260806 09:02:22.321928 19186 heartbeater.cc:464] Registering TS with master...
I20260806 09:02:22.322794 19186 heartbeater.cc:511] Master 127.22.98.62:37257 requested a full tablet report, sending...
I20260806 09:02:22.324447 18582 ts_manager.cc:194] Registered new tserver with Master: 573eb92dab374c5f814fd7d3b5d3c6a0 (127.22.98.3:38679)
I20260806 09:02:22.325364 18582 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.98.3:45817
I20260806 09:02:22.330595 22920 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:02:22.341317 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.335043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.22.98.62:37257)
0806 09:02:22.335318 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.335325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.335629 (+ 304us) server_negotiation.cc:407] Connection header received
0806 09:02:22.335868 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.335873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.335942 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.336028 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:22.336603 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.337315 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.338764 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.339266 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.339384 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.340624 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:22.340629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:22.340633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:22.340654 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:22.340692 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:22.340704 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:22.340785 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.340921 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.341012 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 09:02:22.345911 18582 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:02:22.353098 18582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:02:22.367244 18838 heartbeater.cc:503] Master 127.22.98.62:37257 was elected leader, sending a full tablet report...
I20260806 09:02:22.387878 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.366634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.22.98.2:34405)
0806 09:02:22.378064 (+ 11430us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.378076 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.379318 (+ 1242us) server_negotiation.cc:407] Connection header received
0806 09:02:22.379445 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.379454 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.379764 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.380843 (+ 1079us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:22.381131 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.382682 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.385503 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.386620 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.386780 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.387045 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.387305 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.387487 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11241,"thread_start_us":11107,"threads_started":1}
I20260806 09:02:22.388038 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.366258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.2:34405 (local address 127.0.0.1:40406)
0806 09:02:22.378649 (+ 12391us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.378696 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.378930 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.380041 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.380049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.380087 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:22.380582 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.380598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.382840 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.382845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.385299 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.385314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.386821 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.386827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.387047 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.387110 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.387230 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.387608 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12102,"mutex_wait_us":117,"thread_start_us":11869,"threads_started":1}
I20260806 09:02:22.391748 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.381814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:38679 (local address 127.0.0.1:60978)
0806 09:02:22.382682 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.382701 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.382831 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.383559 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.383565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.383591 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:22.384428 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.384441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.386886 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.386892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.389066 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.389082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.390887 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.390894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.391076 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.391112 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.391201 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.391341 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":6,"thread_start_us":686,"threads_started":1}
I20260806 09:02:22.391839 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.379643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.22.98.3:38679)
0806 09:02:22.382467 (+ 2824us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.382475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.382960 (+ 485us) server_negotiation.cc:407] Connection header received
0806 09:02:22.383064 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.383073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.383319 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.384269 (+ 950us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:22.385107 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.386712 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.389294 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.390692 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.390840 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.391135 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.391268 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.391444 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2638,"thread_start_us":2435,"threads_started":1}
I20260806 09:02:22.392983 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.366262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:41049 (local address 127.0.0.1:58324)
0806 09:02:22.380958 (+ 14696us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.380982 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.381134 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.384812 (+ 3678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.384817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.384843 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:22.385640 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.385655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.387650 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.387660 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.390352 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.390366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.392233 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.392240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.392421 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.392456 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.392522 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.392647 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14522,"mutex_wait_us":189,"thread_start_us":14759,"threads_started":1}
I20260806 09:02:22.392577 18920 tablet_service.cc:1467] Processing CreateTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 (DEFAULT_TABLE table=test-table [id=2484cfbd7fa34f6d82de970f6be1b422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:22.394367 18920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f1fac3ddc24683ac9cf1102a5a8472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:22.394541 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.366733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.22.98.1:41049)
0806 09:02:22.384216 (+ 17483us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.384225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.384253 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:02:22.384341 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.384348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.384591 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.384806 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:22.385967 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.387455 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.390551 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.391982 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.393621 (+ 1639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.393908 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.393985 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.394166 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17308,"thread_start_us":17175,"threads_started":1}
I20260806 09:02:22.397315 19100 tablet_service.cc:1467] Processing CreateTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 (DEFAULT_TABLE table=test-table [id=2484cfbd7fa34f6d82de970f6be1b422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:22.397600 18743 tablet_service.cc:1467] Processing CreateTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 (DEFAULT_TABLE table=test-table [id=2484cfbd7fa34f6d82de970f6be1b422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:02:22.398867 18743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f1fac3ddc24683ac9cf1102a5a8472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:22.398901 19100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f1fac3ddc24683ac9cf1102a5a8472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:22.409946 19209 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Bootstrap starting.
I20260806 09:02:22.414518 19209 tablet_bootstrap.cc:654] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:22.416146 19209 log.cc:824] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:22.416350 19211 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Bootstrap starting.
I20260806 09:02:22.420074 19211 tablet_bootstrap.cc:654] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:22.420463 19210 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Bootstrap starting.
I20260806 09:02:22.422351 19211 log.cc:824] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:22.424558 19210 tablet_bootstrap.cc:654] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:22.424893 19209 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: No bootstrap required, opened a new log
I20260806 09:02:22.425252 19209 ts_tablet_manager.cc:1397] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.001s
I20260806 09:02:22.426033 19210 log.cc:824] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:22.426259 19211 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: No bootstrap required, opened a new log
I20260806 09:02:22.426580 19211 ts_tablet_manager.cc:1397] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260806 09:02:22.429929 19210 tablet_bootstrap.cc:492] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: No bootstrap required, opened a new log
I20260806 09:02:22.430224 19210 ts_tablet_manager.cc:1397] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 09:02:22.437724 19211 raft_consensus.cc:348] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.437724 19209 raft_consensus.cc:348] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.438293 19211 raft_consensus.cc:374] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:22.438293 19209 raft_consensus.cc:374] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:22.438413 19209 raft_consensus.cc:729] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a2031c8ce64ab2a7521af7648c7101, State: Initialized, Role: FOLLOWER
I20260806 09:02:22.438441 19211 raft_consensus.cc:729] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573eb92dab374c5f814fd7d3b5d3c6a0, State: Initialized, Role: FOLLOWER
I20260806 09:02:22.438172 19210 raft_consensus.cc:348] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.438680 19210 raft_consensus.cc:374] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:22.438817 19210 raft_consensus.cc:729] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ba0512970c4dfb8b236142c8684ed3, State: Initialized, Role: FOLLOWER
I20260806 09:02:22.439159 19209 consensus_queue.cc:260] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.439159 19211 consensus_queue.cc:260] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.439671 19210 consensus_queue.cc:260] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.441097 19015 heartbeater.cc:503] Master 127.22.98.62:37257 was elected leader, sending a full tablet report...
I20260806 09:02:22.441098 19186 heartbeater.cc:503] Master 127.22.98.62:37257 was elected leader, sending a full tablet report...
I20260806 09:02:22.441406 19209 ts_tablet_manager.cc:1428] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260806 09:02:22.441670 19211 ts_tablet_manager.cc:1428] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 09:02:22.443271 19210 ts_tablet_manager.cc:1428] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 09:02:22.546783 19217 raft_consensus.cc:482] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:02:22.547163 19217 raft_consensus.cc:504] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.548992 19217 leader_election.cc:288] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6ba0512970c4dfb8b236142c8684ed3 (127.22.98.1:41049), 573eb92dab374c5f814fd7d3b5d3c6a0 (127.22.98.3:38679)
I20260806 09:02:22.556231 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.549490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:34627 (local address 127.22.98.1:41049)
0806 09:02:22.549719 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.549726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.550574 (+ 848us) server_negotiation.cc:407] Connection header received
0806 09:02:22.550738 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.550742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.550820 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.550917 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:22.551600 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.552314 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.554465 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.555272 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.555391 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.555496 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.555858 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.555991 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:02:22.556229 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.549564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.98.2:49423 (local address 127.22.98.3:38679)
0806 09:02:22.549841 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.549847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.550503 (+ 656us) server_negotiation.cc:407] Connection header received
0806 09:02:22.550685 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.550691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.550771 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.550839 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:02:22.551461 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.552333 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.554475 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.555263 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.555388 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.555492 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.555881 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.556002 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 09:02:22.556402 19220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.549585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.1:41049 (local address 127.22.98.2:34627)
0806 09:02:22.550445 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.550463 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.550597 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.550948 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.550953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.550980 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:22.551389 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.551401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.552451 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.552457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.554252 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.554266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.555496 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.555503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.555678 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.555720 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.555794 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.555918 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":94,"thread_start_us":438,"threads_started":1}
I20260806 09:02:22.556430 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.549522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.3:38679 (local address 127.22.98.2:49423)
0806 09:02:22.550367 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0806 09:02:22.550395 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 09:02:22.550539 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:02:22.550880 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:02:22.550885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:02:22.550911 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:02:22.551318 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.551328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.552462 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.552465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.554272 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:02:22.554286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.555514 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.555523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:02:22.555690 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.555737 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 09:02:22.555833 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:02:22.555960 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":106,"thread_start_us":493,"threads_started":1}
W20260806 09:02:22.557044 19187 tablet.cc:2369] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:22.557576 19124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f1fac3ddc24683ac9cf1102a5a8472" candidate_uuid: "62a2031c8ce64ab2a7521af7648c7101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" is_pre_election: true
I20260806 09:02:22.557637 18774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f1fac3ddc24683ac9cf1102a5a8472" candidate_uuid: "62a2031c8ce64ab2a7521af7648c7101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ba0512970c4dfb8b236142c8684ed3" is_pre_election: true
I20260806 09:02:22.558092 19124 raft_consensus.cc:2454] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a2031c8ce64ab2a7521af7648c7101 in term 0.
I20260806 09:02:22.558226 18774 raft_consensus.cc:2454] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a2031c8ce64ab2a7521af7648c7101 in term 0.
I20260806 09:02:22.558857 18863 leader_election.cc:302] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 573eb92dab374c5f814fd7d3b5d3c6a0, 62a2031c8ce64ab2a7521af7648c7101; no voters:
I20260806 09:02:22.559247 19217 raft_consensus.cc:2781] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:02:22.559386 19217 raft_consensus.cc:482] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:02:22.559496 19217 raft_consensus.cc:3037] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:22.561721 19217 raft_consensus.cc:504] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.562369 19217 leader_election.cc:288] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [CANDIDATE]: Term 1 election: Requested vote from peers b6ba0512970c4dfb8b236142c8684ed3 (127.22.98.1:41049), 573eb92dab374c5f814fd7d3b5d3c6a0 (127.22.98.3:38679)
I20260806 09:02:22.562860 18774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f1fac3ddc24683ac9cf1102a5a8472" candidate_uuid: "62a2031c8ce64ab2a7521af7648c7101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ba0512970c4dfb8b236142c8684ed3"
I20260806 09:02:22.562877 19124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f1fac3ddc24683ac9cf1102a5a8472" candidate_uuid: "62a2031c8ce64ab2a7521af7648c7101" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0"
I20260806 09:02:22.563057 18774 raft_consensus.cc:3037] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:22.563060 19124 raft_consensus.cc:3037] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:22.565286 18774 raft_consensus.cc:2454] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a2031c8ce64ab2a7521af7648c7101 in term 1.
I20260806 09:02:22.565290 19124 raft_consensus.cc:2454] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a2031c8ce64ab2a7521af7648c7101 in term 1.
I20260806 09:02:22.565711 18863 leader_election.cc:302] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 573eb92dab374c5f814fd7d3b5d3c6a0, 62a2031c8ce64ab2a7521af7648c7101; no voters:
I20260806 09:02:22.566001 19217 raft_consensus.cc:2781] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:22.566468 19217 raft_consensus.cc:686] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 1 LEADER]: Becoming Leader. State: Replica: 62a2031c8ce64ab2a7521af7648c7101, State: Running, Role: LEADER
I20260806 09:02:22.566948 19217 consensus_queue.cc:237] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } }
I20260806 09:02:22.572376 18581 catalog_manager.cc:5495] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a2031c8ce64ab2a7521af7648c7101 (127.22.98.2). New cstate: current_term: 1 leader_uuid: "62a2031c8ce64ab2a7521af7648c7101" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a2031c8ce64ab2a7521af7648c7101" member_type: VOTER last_known_addr { host: "127.22.98.2" port: 34405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 } health_report { overall_health: UNKNOWN } } }
W20260806 09:02:22.587060 19016 tablet.cc:2369] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:02:22.593247 18839 tablet.cc:2369] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:02:22.634917 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.629297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.22.98.2:34405)
0806 09:02:22.629481 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.629486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.629595 (+ 109us) server_negotiation.cc:407] Connection header received
0806 09:02:22.629804 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.629808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.629905 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.630042 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:02:22.630440 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.631142 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.632618 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.633099 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.633211 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.633795 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:02:22.634359 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:02:22.634495 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.634613 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.634688 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:02:22.643740 18774 raft_consensus.cc:1261] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 1 FOLLOWER]: Refusing update from remote peer 62a2031c8ce64ab2a7521af7648c7101: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:22.643740 19124 raft_consensus.cc:1261] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 1 FOLLOWER]: Refusing update from remote peer 62a2031c8ce64ab2a7521af7648c7101: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:02:22.644835 19221 consensus_queue.cc:1035] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ba0512970c4dfb8b236142c8684ed3" member_type: VOTER last_known_addr { host: "127.22.98.1" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:22.645177 19217 consensus_queue.cc:1035] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [LEADER]: Connected to new peer: Peer: permanent_uuid: "573eb92dab374c5f814fd7d3b5d3c6a0" member_type: VOTER last_known_addr { host: "127.22.98.3" port: 38679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:02:22.657157 19225 mvcc.cc:205] Tried to move back new op lower bound from 7315484437058060288 to 7315484436758462464. Current Snapshot: MvccSnapshot[applied={T|T < 7315484437058060288}]
I20260806 09:02:22.672315 18582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:22.672605 18582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:02:22.676707 18582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 152321f845474b1e9a4650cb4c19a90a: Sending DeleteTablet for 3 replicas of tablet 44f1fac3ddc24683ac9cf1102a5a8472
I20260806 09:02:22.678078 18743 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:22 UTC) from {username='slave'} at 127.0.0.1:58324
I20260806 09:02:22.678198 19100 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:22 UTC) from {username='slave'} at 127.0.0.1:60978
I20260806 09:02:22.679179 18920 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f1fac3ddc24683ac9cf1102a5a8472 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:02:22 UTC) from {username='slave'} at 127.0.0.1:40406
I20260806 09:02:22.679412 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 18665
I20260806 09:02:22.683318 19243 tablet_replica.cc:330] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: stopping tablet replica
I20260806 09:02:22.685189 19243 raft_consensus.cc:2229] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:02:22.689354 19241 tablet_replica.cc:330] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: stopping tablet replica
I20260806 09:02:22.689839 19243 raft_consensus.cc:2258] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:22.690081 19241 raft_consensus.cc:2229] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:22.690667 19241 raft_consensus.cc:2258] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:22.691289 19240 tablet_replica.cc:330] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: stopping tablet replica
I20260806 09:02:22.691787 19240 raft_consensus.cc:2229] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:02:22.692305 19240 raft_consensus.cc:2258] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:02:22.692725 19243 ts_tablet_manager.cc:1905] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:22.694875 19240 ts_tablet_manager.cc:1905] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:22.694869 19241 ts_tablet_manager.cc:1905] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:02:22.702108 19240 ts_tablet_manager.cc:1918] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:22.702145 19241 ts_tablet_manager.cc:1918] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:22.702185 19243 ts_tablet_manager.cc:1918] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:02:22.702358 19243 log.cc:1196] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-1/wal/wals/44f1fac3ddc24683ac9cf1102a5a8472
I20260806 09:02:22.702407 19240 log.cc:1196] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-0/wal/wals/44f1fac3ddc24683ac9cf1102a5a8472
I20260806 09:02:22.702433 19241 log.cc:1196] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Deleting WAL directory at /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786006610327111-22920-0/minicluster-data/ts-2/wal/wals/44f1fac3ddc24683ac9cf1102a5a8472
I20260806 09:02:22.702993 19240 ts_tablet_manager.cc:1939] T 44f1fac3ddc24683ac9cf1102a5a8472 P b6ba0512970c4dfb8b236142c8684ed3: Deleting consensus metadata
I20260806 09:02:22.703020 19241 ts_tablet_manager.cc:1939] T 44f1fac3ddc24683ac9cf1102a5a8472 P 573eb92dab374c5f814fd7d3b5d3c6a0: Deleting consensus metadata
I20260806 09:02:22.704982 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.680727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.22.98.1:41049)
0806 09:02:22.681096 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.681104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.682699 (+ 1595us) server_negotiation.cc:407] Connection header received
0806 09:02:22.683806 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.683813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.683939 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.684020 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:22.685567 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.686888 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.693153 (+ 6265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.693859 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.693978 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.703116 (+ 9138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:22.703132 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:22.703153 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:22.704005 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:22.704071 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:22.704105 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:22.704199 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.704497 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.704629 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260806 09:02:22.705035 18564 catalog_manager.cc:4841] TS 573eb92dab374c5f814fd7d3b5d3c6a0 (127.22.98.3:38679): tablet 44f1fac3ddc24683ac9cf1102a5a8472 (table test-table [id=2484cfbd7fa34f6d82de970f6be1b422]) successfully deleted
I20260806 09:02:22.705286 18565 catalog_manager.cc:4841] TS b6ba0512970c4dfb8b236142c8684ed3 (127.22.98.1:41049): tablet 44f1fac3ddc24683ac9cf1102a5a8472 (table test-table [id=2484cfbd7fa34f6d82de970f6be1b422]) successfully deleted
I20260806 09:02:22.705480 19243 ts_tablet_manager.cc:1939] T 44f1fac3ddc24683ac9cf1102a5a8472 P 62a2031c8ce64ab2a7521af7648c7101: Deleting consensus metadata
I20260806 09:02:22.706765 18833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:22.707088 18564 catalog_manager.cc:4841] TS 62a2031c8ce64ab2a7521af7648c7101 (127.22.98.2:34405): tablet 44f1fac3ddc24683ac9cf1102a5a8472 (table test-table [id=2484cfbd7fa34f6d82de970f6be1b422]) successfully deleted
I20260806 09:02:22.797132 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 18665
I20260806 09:02:22.829259 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 18842
I20260806 09:02:22.837321 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.830084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.22.98.2:34405)
0806 09:02:22.830291 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.830301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.830325 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:02:22.830533 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.830537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.830664 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.830765 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:22.831169 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.832023 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.833557 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.834264 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.834409 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.835636 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:22.835648 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:22.835666 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:22.836469 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:22.836537 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:22.836563 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:22.836679 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.836824 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.836932 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:02:22.838084 19010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:22.950611 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 18842
I20260806 09:02:22.973308 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19018
I20260806 09:02:22.980263 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:22.974148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.22.98.3:38679)
0806 09:02:22.974355 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:22.974360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:22.974410 (+ 50us) server_negotiation.cc:407] Connection header received
0806 09:02:22.974574 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:22.974578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:22.974675 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:22.974760 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:22.975105 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.975886 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.977247 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:22.977676 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:22.977798 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:22.978760 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:22.978770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:22.978787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:22.979390 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:22.979451 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:22.979474 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:22.979620 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:22.979804 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 09:02:22.979907 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:02:22.980966 19180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:23.069774 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19018
I20260806 09:02:23.091681 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 18544
I20260806 09:02:23.093065 18619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:23.203727 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 18544
2026-08-06T09:02:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3038 ms)
[----------] 6 tests from EncryptionPolicyTest (23984 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:23 dist-test-slave-jpj1 krb5kdc[19251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:23 dist-test-slave-jpj1 krb5kdc[19251](info): set up 2 sockets
Aug 06 09:02:23 dist-test-slave-jpj1 krb5kdc[19251](info): commencing operation
krb5kdc: starting...
W20260806 09:02:24.430298 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:24 dist-test-slave-jpj1 krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:24.763000 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:25.119076 19269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:25.119410 19269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:25.119508 19269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:25.127061 19269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:25.127197 19269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:25.127246 19269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:25.127310 19269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:25.127353 19269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:25.138114 19269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:25.139573 19269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:25.140990 19269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:25.153376 19274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:25.153461 19278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:25.153429 19275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:25.155002 19269 server_base.cc:1034] running on GCE node
I20260806 09:02:25.155951 19269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:25.157512 19269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:25.158730 19269 hybrid_clock.cc:648] HybridClock initialized: now 1786006945158686 us; error 37 us; skew 500 ppm
I20260806 09:02:25.159118 19269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:25 dist-test-slave-jpj1 krb5kdc[19251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006945, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:25.166421 19269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:25.168233 19269 webserver.cc:460] Webserver started at http://127.0.0.1:42007/ using document root <none> and password file <none>
I20260806 09:02:25.168972 19269 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:25.169098 19269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:25.169441 19269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:25.172676 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "6ecb31dd3001477a8b9b09df0dcbfd56"
format_stamp: "Formatted at 2026-08-06 09:02:25 on dist-test-slave-jpj1"
server_key: "c325533dcaf634b3aa336c50f0778204"
server_key_iv: "9265b53858f490b5ba3fb8edbeb87f7c"
server_key_version: "encryptionkey@0"
I20260806 09:02:25.173506 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "6ecb31dd3001477a8b9b09df0dcbfd56"
format_stamp: "Formatted at 2026-08-06 09:02:25 on dist-test-slave-jpj1"
server_key: "c325533dcaf634b3aa336c50f0778204"
server_key_iv: "9265b53858f490b5ba3fb8edbeb87f7c"
server_key_version: "encryptionkey@0"
I20260806 09:02:25.179316 19269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:02:25.183341 19288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:25.184927 19269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:25.185118 19269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "6ecb31dd3001477a8b9b09df0dcbfd56"
format_stamp: "Formatted at 2026-08-06 09:02:25 on dist-test-slave-jpj1"
server_key: "c325533dcaf634b3aa336c50f0778204"
server_key_iv: "9265b53858f490b5ba3fb8edbeb87f7c"
server_key_version: "encryptionkey@0"
I20260806 09:02:25.185323 19269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:25.204133 19269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:25.208668 19269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:25.208966 19269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:25.228369 19269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37469
I20260806 09:02:25.228413 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37469 every 8 connection(s)
I20260806 09:02:25.229887 19269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:25.234191 19370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:25.236336 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19269
I20260806 09:02:25.236569 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:25.237439 22920 external_mini_cluster.cc:1428] Setting key e90f7917e0dc1e998019467ada5da82e
I20260806 09:02:25.247915 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56: Bootstrap starting.
I20260806 09:02:25.252307 19370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56: Neither blocks nor log segments found. Creating new log.
Aug 06 09:02:25 dist-test-slave-jpj1 krb5kdc[19251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:02:25.253588 19370 log.cc:824] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:25.257256 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56: No bootstrap required, opened a new log
I20260806 09:02:25.263963 19374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:25.239701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38707 (local address 127.0.0.1:37469)
0806 09:02:25.240561 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:25.240573 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:25.240619 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:02:25.242397 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:25.242424 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:25.242847 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:25.243186 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:25.245217 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:25.246638 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:25.248616 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:25.249332 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:25.249482 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:25.254867 (+ 5385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:25.254883 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:25.254907 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:25.254969 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:25.258216 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:25.259437 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:25.259446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:25.259453 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:25.259803 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:25.260414 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:25.260421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:25.260424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:25.261110 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:25.261407 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:25.261985 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 09:02:25.262585 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":388,"threads_started":1}
I20260806 09:02:25.267165 19370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } }
I20260806 09:02:25.267700 19370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:25.267815 19370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ecb31dd3001477a8b9b09df0dcbfd56, State: Initialized, Role: FOLLOWER
I20260806 09:02:25.268469 19370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } }
I20260806 09:02:25.268761 19370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:25.268908 19370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:25.268949 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:25.269095 19370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:25.272445 19370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } }
I20260806 09:02:25.273146 19370 leader_election.cc:302] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ecb31dd3001477a8b9b09df0dcbfd56; no voters:
I20260806 09:02:25.273850 19370 leader_election.cc:288] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:25.273950 19384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:25.276050 19384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [term 1 LEADER]: Becoming Leader. State: Replica: 6ecb31dd3001477a8b9b09df0dcbfd56, State: Running, Role: LEADER
I20260806 09:02:25.276397 19370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:25.276769 19384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } }
I20260806 09:02:25.279795 19385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } } }
I20260806 09:02:25.280257 19385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:25.280798 19386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ecb31dd3001477a8b9b09df0dcbfd56. Latest consensus state: current_term: 1 leader_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecb31dd3001477a8b9b09df0dcbfd56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37469 } } }
I20260806 09:02:25.281112 19386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:25.291446 19398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:25.294831 19374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:25.274928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58777 (local address 127.0.0.1:37469)
0806 09:02:25.275125 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:25.275131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:25.276022 (+ 891us) server_negotiation.cc:407] Connection header received
0806 09:02:25.276403 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:25.276409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:25.276497 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:25.276616 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:25.278591 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:25.279848 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:25.281662 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:25.282364 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:25.282471 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:25.286439 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:25.286446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:25.286452 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:25.286505 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:25.290851 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:25.291985 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:25.291991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:25.291994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:25.292142 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:25.292789 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:25.292794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:25.292797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:25.293245 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:25.293505 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:25.294254 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 09:02:25.294459 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 09:02:25.297871 19398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:25.308564 19398 catalog_manager.cc:1347] Generated new cluster ID: 6c2d03b7e8f040c58e1a89bf724f323a
I20260806 09:02:25.308704 19398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:25.342798 19398 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:25.344255 19398 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:25.356343 19398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ecb31dd3001477a8b9b09df0dcbfd56: Generated new TSK 0
I20260806 09:02:25.357532 19398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:25.369714 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19269
I20260806 09:02:25.371421 19361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:25.470057 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19269
W20260806 09:02:25.494943 19249 connection.cc:535] client connection to 127.0.0.1:37469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37469): Connection reset by peer (error 104)
2026-08-06T09:02:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2257 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:25 dist-test-slave-jpj1 krb5kdc[19422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:25 dist-test-slave-jpj1 krb5kdc[19422](info): set up 2 sockets
Aug 06 09:02:25 dist-test-slave-jpj1 krb5kdc[19422](info): commencing operation
krb5kdc: starting...
W20260806 09:02:26.685813 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:26 dist-test-slave-jpj1 krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:27.012322 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:27.376529 19441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:27.376837 19441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:27.376972 19441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:27.384274 19441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:27.384402 19441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:27.384479 19441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:27.384562 19441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:27.384621 19441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:27.394917 19441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:27.396327 19441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:27.397681 19441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:27.411191 19451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:27.411278 19455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:27.412251 19453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:27.412642 19441 server_base.cc:1034] running on GCE node
I20260806 09:02:27.413543 19441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:27.415084 19441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:27.416337 19441 hybrid_clock.cc:648] HybridClock initialized: now 1786006947416277 us; error 43 us; skew 500 ppm
I20260806 09:02:27.416745 19441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:27 dist-test-slave-jpj1 krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006947, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:27.420332 19441 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:27.422072 19441 webserver.cc:460] Webserver started at http://127.0.0.1:44605/ using document root <none> and password file <none>
I20260806 09:02:27.422854 19441 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:27.422993 19441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:27.423322 19441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:27.426680 19441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d"
format_stamp: "Formatted at 2026-08-06 09:02:27 on dist-test-slave-jpj1"
server_key: "3b799b885bed88d9e47585a45fc68c4c"
server_key_iv: "05559cacf98b0d9311f60f4b36077bf9"
server_key_version: "encryptionkey@0"
I20260806 09:02:27.427587 19441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d"
format_stamp: "Formatted at 2026-08-06 09:02:27 on dist-test-slave-jpj1"
server_key: "3b799b885bed88d9e47585a45fc68c4c"
server_key_iv: "05559cacf98b0d9311f60f4b36077bf9"
server_key_version: "encryptionkey@0"
I20260806 09:02:27.433600 19441 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 09:02:27.437875 19463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:27.439715 19441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:27.439940 19441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d"
format_stamp: "Formatted at 2026-08-06 09:02:27 on dist-test-slave-jpj1"
server_key: "3b799b885bed88d9e47585a45fc68c4c"
server_key_iv: "05559cacf98b0d9311f60f4b36077bf9"
server_key_version: "encryptionkey@0"
I20260806 09:02:27.440222 19441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:27.475356 19441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:27.480880 19441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:27.481205 19441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:27.506491 19537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34307 every 8 connection(s)
I20260806 09:02:27.506491 19441 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34307
I20260806 09:02:27.508605 19441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:27.514400 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:27.517395 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19441
I20260806 09:02:27.517596 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:27.518108 22920 external_mini_cluster.cc:1428] Setting key 1153b1a271c7a2f3ce5faf8e75eca666
I20260806 09:02:27.530064 19538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d: Bootstrap starting.
I20260806 09:02:27.534801 19538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:27.535948 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:27.520515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45397 (local address 127.0.0.1:34307)
0806 09:02:27.521586 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:27.521604 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:27.521667 (+ 63us) server_negotiation.cc:407] Connection header received
0806 09:02:27.524190 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:27.524225 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:27.524826 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:27.525370 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:27.525974 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:27.528139 (+ 2165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:27.529777 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:27.530929 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:27.531126 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:27.532838 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:27.532851 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:27.532872 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:27.533632 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:27.533750 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:27.533789 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:27.533994 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:27.534248 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 09:02:27.534451 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":380,"threads_started":1}
I20260806 09:02:27.536372 19538 log.cc:824] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:27.540313 19538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d: No bootstrap required, opened a new log
I20260806 09:02:27.541560 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:27.550015 19538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } }
I20260806 09:02:27.550807 19538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:27.550956 19538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e574f5d64684bc2aa1fe18fbf1ecd3d, State: Initialized, Role: FOLLOWER
I20260806 09:02:27.551779 19538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } }
I20260806 09:02:27.552062 19538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:27.552223 19538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:27.552402 19538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:27.554513 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:27.546845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40627 (local address 127.0.0.1:34307)
0806 09:02:27.547158 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:27.547165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:27.547573 (+ 408us) server_negotiation.cc:407] Connection header received
0806 09:02:27.547819 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:27.547824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:27.547907 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:27.548016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:27.548700 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:27.549514 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:27.551682 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:27.552275 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:27.552433 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:27.553730 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:27.553737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:27.553740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:27.553776 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:27.553828 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:27.553845 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:27.553965 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:27.554177 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 09:02:27.554238 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:02:27.555301 19538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } }
I20260806 09:02:27.555908 19538 leader_election.cc:302] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e574f5d64684bc2aa1fe18fbf1ecd3d; no voters:
I20260806 09:02:27.556622 19538 leader_election.cc:288] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:27.556757 19553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:27.557474 19553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [term 1 LEADER]: Becoming Leader. State: Replica: 9e574f5d64684bc2aa1fe18fbf1ecd3d, State: Running, Role: LEADER
I20260806 09:02:27.558386 19553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } }
I20260806 09:02:27.559437 19538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:27.564255 19554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } } }
I20260806 09:02:27.564824 19554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:27.564839 19555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e574f5d64684bc2aa1fe18fbf1ecd3d. Latest consensus state: current_term: 1 leader_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e574f5d64684bc2aa1fe18fbf1ecd3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34307 } } }
I20260806 09:02:27.565629 19555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:27.566758 19563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:27.571802 19563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:27.581646 19563 catalog_manager.cc:1347] Generated new cluster ID: 0eb2d3cfdc6549dfb598b02a5838d604
I20260806 09:02:27.581799 19563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:27.614807 19563 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:27.616281 19563 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:27.631042 19563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e574f5d64684bc2aa1fe18fbf1ecd3d: Generated new TSK 0
I20260806 09:02:27.632009 19563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:27.641340 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19441
W20260806 09:02:27.642520 19533 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45397
W20260806 09:02:27.643345 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19441: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:27.643524 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19441
W20260806 09:02:27.667110 19420 connection.cc:535] client connection to 127.0.0.1:34307 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34307): Connection reset by peer (error 104)
2026-08-06T09:02:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2171 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:27 dist-test-slave-jpj1 krb5kdc[19579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:27 dist-test-slave-jpj1 krb5kdc[19579](info): set up 2 sockets
Aug 06 09:02:27 dist-test-slave-jpj1 krb5kdc[19579](info): commencing operation
krb5kdc: starting...
W20260806 09:02:28.866137 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:29 dist-test-slave-jpj1 krb5kdc[19579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:29.196614 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:29.564409 19596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:29.564738 19596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:29.564864 19596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:29.572281 19596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:29.572413 19596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:29.572494 19596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:29.572582 19596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:29.572635 19596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:29.582873 19596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:29.584290 19596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:29.585616 19596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:29.597820 19604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:29.597868 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:29.598237 19609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:29.598819 19596 server_base.cc:1034] running on GCE node
I20260806 09:02:29.599704 19596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:29.601195 19596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:29.602427 19596 hybrid_clock.cc:648] HybridClock initialized: now 1786006949602377 us; error 35 us; skew 500 ppm
I20260806 09:02:29.602780 19596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:29 dist-test-slave-jpj1 krb5kdc[19579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:29.606004 19596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:29.607643 19596 webserver.cc:460] Webserver started at http://127.0.0.1:42893/ using document root <none> and password file <none>
I20260806 09:02:29.608345 19596 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:29.608467 19596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:29.608770 19596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:29.612175 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "978e9cca4ef04001b765113cdc57e75d"
format_stamp: "Formatted at 2026-08-06 09:02:29 on dist-test-slave-jpj1"
server_key: "d1fdefb6b17a2f8cc365e9008d78d401"
server_key_iv: "bddbb0435a0d9cd9d718ec899d1a87b8"
server_key_version: "encryptionkey@0"
I20260806 09:02:29.613292 19596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "978e9cca4ef04001b765113cdc57e75d"
format_stamp: "Formatted at 2026-08-06 09:02:29 on dist-test-slave-jpj1"
server_key: "d1fdefb6b17a2f8cc365e9008d78d401"
server_key_iv: "bddbb0435a0d9cd9d718ec899d1a87b8"
server_key_version: "encryptionkey@0"
I20260806 09:02:29.619416 19596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 09:02:29.623232 19619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:29.624985 19596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:29.625180 19596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "978e9cca4ef04001b765113cdc57e75d"
format_stamp: "Formatted at 2026-08-06 09:02:29 on dist-test-slave-jpj1"
server_key: "d1fdefb6b17a2f8cc365e9008d78d401"
server_key_iv: "bddbb0435a0d9cd9d718ec899d1a87b8"
server_key_version: "encryptionkey@0"
I20260806 09:02:29.625391 19596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:29.631289 19596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:29.635578 19596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:29.635885 19596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:29.654968 19596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32847
I20260806 09:02:29.654997 19685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32847 every 8 connection(s)
I20260806 09:02:29.656481 19596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:29.658809 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19596
I20260806 09:02:29.658967 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:29.659428 22920 external_mini_cluster.cc:1428] Setting key fbd7c59c9b5005a6e94fc32aa752fe2b
I20260806 09:02:29.660925 19686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:29.667367 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:29.661290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45175 (local address 127.0.0.1:32847)
0806 09:02:29.662108 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:29.662122 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:29.662166 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:02:29.663831 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:29.663854 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:29.664212 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:29.664548 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:29.665720 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:29.665728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:29.665741 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:29.665767 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:29.665852 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:29.665872 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:29.666060 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:29.666176 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 09:02:29.666360 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":356,"threads_started":1}
I20260806 09:02:29.671197 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:29.673802 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d: Bootstrap starting.
I20260806 09:02:29.677287 19686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:29.678186 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:29.675622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57087 (local address 127.0.0.1:32847)
0806 09:02:29.675831 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:29.675839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:29.676308 (+ 469us) server_negotiation.cc:407] Connection header received
0806 09:02:29.676476 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:29.676481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:29.676531 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:29.676616 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:29.677542 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:29.677549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:29.677554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:29.677570 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:29.677620 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:29.677635 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:29.677742 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:29.677850 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 09:02:29.677908 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:02:29.678537 19686 log.cc:824] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:29.681461 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d: No bootstrap required, opened a new log
I20260806 09:02:29.689091 19686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260806 09:02:29.689553 19686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:29.689670 19686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978e9cca4ef04001b765113cdc57e75d, State: Initialized, Role: FOLLOWER
I20260806 09:02:29.690282 19686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260806 09:02:29.690524 19686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:29.690656 19686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:29.690824 19686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:29.693177 19686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260806 09:02:29.693727 19686 leader_election.cc:302] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 978e9cca4ef04001b765113cdc57e75d; no voters:
I20260806 09:02:29.694352 19686 leader_election.cc:288] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:29.694458 19696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:29.695155 19696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [term 1 LEADER]: Becoming Leader. State: Replica: 978e9cca4ef04001b765113cdc57e75d, State: Running, Role: LEADER
I20260806 09:02:29.695807 19696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260806 09:02:29.696576 19686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:29.698751 19698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 978e9cca4ef04001b765113cdc57e75d. Latest consensus state: current_term: 1 leader_uuid: "978e9cca4ef04001b765113cdc57e75d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } } }
I20260806 09:02:29.699028 19697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "978e9cca4ef04001b765113cdc57e75d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978e9cca4ef04001b765113cdc57e75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } } }
I20260806 09:02:29.699229 19698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:29.699299 19697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:29.702250 19706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:29.708288 19706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:29.717582 19706 catalog_manager.cc:1347] Generated new cluster ID: f435a53fe53844aebf35ab5ffa636f20
I20260806 09:02:29.717718 19706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:29.737319 19706 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:29.737910 19706 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:29.746721 19706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 978e9cca4ef04001b765113cdc57e75d: Generated new TSK 0
I20260806 09:02:29.747931 19706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:29.756522 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19596
W20260806 09:02:29.757800 19679 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45175
W20260806 09:02:29.758628 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19596: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:29.758800 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19596
2026-08-06T09:02:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2119 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:29 dist-test-slave-jpj1 krb5kdc[19725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:29 dist-test-slave-jpj1 krb5kdc[19725](info): set up 2 sockets
Aug 06 09:02:29 dist-test-slave-jpj1 krb5kdc[19725](info): commencing operation
krb5kdc: starting...
W20260806 09:02:30.972743 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:31 dist-test-slave-jpj1 krb5kdc[19725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:31.304541 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:31.667732 19743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:31.668052 19743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:31.668174 19743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:31.675971 19743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:31.676100 19743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:31.676160 19743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:31.676252 19743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:31.676321 19743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:31.686527 19743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:31.687965 19743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:31.689388 19743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:31.701942 19750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:31.701964 19751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:31.702013 19754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:31.702939 19743 server_base.cc:1034] running on GCE node
I20260806 09:02:31.703899 19743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:31.705404 19743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:31.706660 19743 hybrid_clock.cc:648] HybridClock initialized: now 1786006951706557 us; error 52 us; skew 500 ppm
I20260806 09:02:31.707023 19743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:31 dist-test-slave-jpj1 krb5kdc[19725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006951, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:31.714754 19743 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:31.716651 19743 webserver.cc:460] Webserver started at http://127.0.0.1:43811/ using document root <none> and password file <none>
I20260806 09:02:31.717398 19743 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:31.717527 19743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:31.717869 19743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:31.721172 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "824c563cc21f49eda4b6c47557d770fd"
format_stamp: "Formatted at 2026-08-06 09:02:31 on dist-test-slave-jpj1"
server_key: "c5ed7f5cc891186b196870d4a2dd6d96"
server_key_iv: "b3c7aab2c0e43c0e16e690568707a8c4"
server_key_version: "encryptionkey@0"
I20260806 09:02:31.722049 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "824c563cc21f49eda4b6c47557d770fd"
format_stamp: "Formatted at 2026-08-06 09:02:31 on dist-test-slave-jpj1"
server_key: "c5ed7f5cc891186b196870d4a2dd6d96"
server_key_iv: "b3c7aab2c0e43c0e16e690568707a8c4"
server_key_version: "encryptionkey@0"
I20260806 09:02:31.727886 19743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:02:31.731868 19765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:31.733448 19743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:31.733669 19743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "824c563cc21f49eda4b6c47557d770fd"
format_stamp: "Formatted at 2026-08-06 09:02:31 on dist-test-slave-jpj1"
server_key: "c5ed7f5cc891186b196870d4a2dd6d96"
server_key_iv: "b3c7aab2c0e43c0e16e690568707a8c4"
server_key_version: "encryptionkey@0"
I20260806 09:02:31.733882 19743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:31.755971 19743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:31.761292 19743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:31.761623 19743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:31.781214 19743 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38599
I20260806 09:02:31.781245 19839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38599 every 8 connection(s)
I20260806 09:02:31.782728 19743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:31.786888 19840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:31.787457 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19743
I20260806 09:02:31.787675 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:31.788268 22920 external_mini_cluster.cc:1428] Setting key efc75576e2bb324133425afe88f747bc
I20260806 09:02:31.802229 19840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd: Bootstrap starting.
Aug 06 09:02:31 dist-test-slave-jpj1 krb5kdc[19725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:02:31.806514 19840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:31.807750 19840 log.cc:824] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:31.811439 19840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd: No bootstrap required, opened a new log
I20260806 09:02:31.814757 19844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:31.790357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43063 (local address 127.0.0.1:38599)
0806 09:02:31.791226 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:31.791242 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:31.791290 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:02:31.793641 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:31.793680 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:31.794215 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:31.794608 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:31.796620 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:31.798075 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:31.799973 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:31.800723 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:31.800881 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:31.806197 (+ 5316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:31.806210 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:31.806227 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:31.806277 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:31.809877 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:31.810964 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:31.810973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:31.810980 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:31.811181 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:31.811687 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:31.811692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:31.811694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:31.812330 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:31.812590 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:31.813142 (+ 552us) server_negotiation.cc:299] Negotiation successful
0806 09:02:31.813593 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":394,"threads_started":1}
I20260806 09:02:31.819418 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:31.820986 19840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 09:02:31.821466 19840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:31.821568 19840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824c563cc21f49eda4b6c47557d770fd, State: Initialized, Role: FOLLOWER
I20260806 09:02:31.822345 19840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 09:02:31.822677 19840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:31.822836 19840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:31.823038 19840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:31.826297 19840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 09:02:31.827024 19840 leader_election.cc:302] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824c563cc21f49eda4b6c47557d770fd; no voters:
I20260806 09:02:31.827790 19840 leader_election.cc:288] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:31.827881 19852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:31.829105 19852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [term 1 LEADER]: Becoming Leader. State: Replica: 824c563cc21f49eda4b6c47557d770fd, State: Running, Role: LEADER
I20260806 09:02:31.829824 19852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 09:02:31.830384 19840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:31.832787 19855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824c563cc21f49eda4b6c47557d770fd. Latest consensus state: current_term: 1 leader_uuid: "824c563cc21f49eda4b6c47557d770fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } } }
I20260806 09:02:31.833652 19855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:31.832765 19854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "824c563cc21f49eda4b6c47557d770fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824c563cc21f49eda4b6c47557d770fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } } }
I20260806 09:02:31.833796 19854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:31.841398 19864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:31.843886 19844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:31.824998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47229 (local address 127.0.0.1:38599)
0806 09:02:31.825213 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:31.825220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:31.825762 (+ 542us) server_negotiation.cc:407] Connection header received
0806 09:02:31.825947 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:31.825951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:31.826029 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:31.826119 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:31.828187 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:31.829444 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:31.831349 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:31.832152 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:31.832313 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:31.836107 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:31.836116 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:31.836122 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:31.836174 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:31.840300 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:31.841276 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:31.841286 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:31.841291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:31.841434 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:31.841945 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:31.841951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:31.841954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:31.842385 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:31.842692 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:31.843075 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 09:02:31.843313 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:02:31.847970 19864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:31.859402 19864 catalog_manager.cc:1347] Generated new cluster ID: b321575e4a3b40058ce6f20cea46954b
I20260806 09:02:31.859587 19864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:31.872962 19864 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:31.874329 19864 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:31.884716 19864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 824c563cc21f49eda4b6c47557d770fd: Generated new TSK 0
I20260806 09:02:31.885660 19864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:31.893424 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19743
I20260806 09:02:31.894811 19830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:31.987754 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19743
2026-08-06T09:02:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:32 dist-test-slave-jpj1 krb5kdc[19882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:32 dist-test-slave-jpj1 krb5kdc[19882](info): set up 2 sockets
Aug 06 09:02:32 dist-test-slave-jpj1 krb5kdc[19882](info): commencing operation
krb5kdc: starting...
W20260806 09:02:33.219513 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:33 dist-test-slave-jpj1 krb5kdc[19882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:33.559077 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:33.913515 19898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:33.913825 19898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:33.913950 19898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:33.921219 19898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:33.921334 19898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:33.921416 19898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:33.921494 19898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:33.921540 19898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:33.931656 19898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:33.933065 19898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:33.934489 19898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:33.947187 19905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:33.947150 19913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:33.947162 19907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:33.947680 19898 server_base.cc:1034] running on GCE node
I20260806 09:02:33.948737 19898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:33.950294 19898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:33.951520 19898 hybrid_clock.cc:648] HybridClock initialized: now 1786006953951463 us; error 39 us; skew 500 ppm
I20260806 09:02:33.951977 19898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:33 dist-test-slave-jpj1 krb5kdc[19882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:33.955686 19898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:33.957262 19898 webserver.cc:460] Webserver started at http://127.0.0.1:43003/ using document root <none> and password file <none>
I20260806 09:02:33.957924 19898 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:33.958031 19898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:33.958302 19898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:33.961418 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "2ae910b9de1a40dda2bd14b216852cd4"
format_stamp: "Formatted at 2026-08-06 09:02:33 on dist-test-slave-jpj1"
server_key: "32404432252366c9ff432a37a4a3156d"
server_key_iv: "5f7fe30a6a1113039de230c200f4656c"
server_key_version: "encryptionkey@0"
I20260806 09:02:33.962199 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "2ae910b9de1a40dda2bd14b216852cd4"
format_stamp: "Formatted at 2026-08-06 09:02:33 on dist-test-slave-jpj1"
server_key: "32404432252366c9ff432a37a4a3156d"
server_key_iv: "5f7fe30a6a1113039de230c200f4656c"
server_key_version: "encryptionkey@0"
I20260806 09:02:33.967708 19898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:02:33.972009 19923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:33.973730 19898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:02:33.973985 19898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "2ae910b9de1a40dda2bd14b216852cd4"
format_stamp: "Formatted at 2026-08-06 09:02:33 on dist-test-slave-jpj1"
server_key: "32404432252366c9ff432a37a4a3156d"
server_key_iv: "5f7fe30a6a1113039de230c200f4656c"
server_key_version: "encryptionkey@0"
I20260806 09:02:33.974220 19898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:33.990010 19898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:33.994546 19898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:33.994848 19898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:34.014416 19898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33815
I20260806 09:02:34.014449 19998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33815 every 8 connection(s)
I20260806 09:02:34.015939 19898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:34.020011 19999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:34.021605 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 19898
I20260806 09:02:34.021807 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:34.022377 22920 external_mini_cluster.cc:1428] Setting key 186a6e180f094ce3d569001d8e893f47
I20260806 09:02:34.034729 19999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4: Bootstrap starting.
I20260806 09:02:34.036396 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:34.024292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54979 (local address 127.0.0.1:33815)
0806 09:02:34.025175 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:34.025188 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:34.025234 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:02:34.026914 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:34.026945 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:34.027375 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:34.027801 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:34.028415 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:34.029765 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:34.031719 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:34.032466 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:34.032600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:34.033854 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:34.033863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:34.033879 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:34.034457 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:34.034536 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:34.034563 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:34.034650 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:34.034982 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 09:02:34.035267 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":434,"threads_started":1}
I20260806 09:02:34.038160 19999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:34.039355 19999 log.cc:824] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:34.040194 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:34.042136 19999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4: No bootstrap required, opened a new log
I20260806 09:02:34.052215 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:34.044447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49921 (local address 127.0.0.1:33815)
0806 09:02:34.044648 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:34.044655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:34.045113 (+ 458us) server_negotiation.cc:407] Connection header received
0806 09:02:34.045272 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:34.045281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:34.045365 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:34.045494 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:34.046047 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:34.047153 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:34.049056 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:34.049795 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:34.049945 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:34.051431 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:34.051437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:34.051440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:34.051461 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:34.051505 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:34.051521 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:34.051626 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:34.051757 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 09:02:34.051843 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:02:34.051877 19999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } }
I20260806 09:02:34.052412 19999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:34.052546 19999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae910b9de1a40dda2bd14b216852cd4, State: Initialized, Role: FOLLOWER
I20260806 09:02:34.053185 19999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } }
I20260806 09:02:34.053429 19999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:34.053581 19999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:34.053773 19999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:34.056336 19999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } }
I20260806 09:02:34.056907 19999 leader_election.cc:302] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ae910b9de1a40dda2bd14b216852cd4; no voters:
I20260806 09:02:34.057548 19999 leader_election.cc:288] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:34.057683 20020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:34.058473 20020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ae910b9de1a40dda2bd14b216852cd4, State: Running, Role: LEADER
I20260806 09:02:34.059028 20020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } }
I20260806 09:02:34.059784 19999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:34.062391 20022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ae910b9de1a40dda2bd14b216852cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } } }
I20260806 09:02:34.062858 20022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:34.063217 20024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ae910b9de1a40dda2bd14b216852cd4. Latest consensus state: current_term: 1 leader_uuid: "2ae910b9de1a40dda2bd14b216852cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae910b9de1a40dda2bd14b216852cd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33815 } } }
I20260806 09:02:34.063654 20024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:34.064982 20032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:34.070273 20032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:34.081177 20032 catalog_manager.cc:1347] Generated new cluster ID: 30bc716558bf4e1bb77700d890ecfe70
I20260806 09:02:34.081326 20032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:34.106600 20032 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:34.108084 20032 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:34.116930 20032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ae910b9de1a40dda2bd14b216852cd4: Generated new TSK 0
I20260806 09:02:34.117872 20032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:34.129994 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19898
W20260806 09:02:34.131364 19991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54979
W20260806 09:02:34.132158 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 19898: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:34.132344 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 19898
2026-08-06T09:02:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:34 dist-test-slave-jpj1 krb5kdc[20047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:34 dist-test-slave-jpj1 krb5kdc[20047](info): set up 2 sockets
Aug 06 09:02:34 dist-test-slave-jpj1 krb5kdc[20047](info): commencing operation
krb5kdc: starting...
W20260806 09:02:35.357652 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:35 dist-test-slave-jpj1 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:35.698768 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:36.056879 20069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:36.057199 20069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:36.057324 20069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:36.064797 20069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:36.064921 20069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:36.064996 20069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:36.065080 20069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:36.065146 20069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:36.075727 20069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:36.077164 20069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:36.078565 20069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:36.091310 20074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:36.091279 20078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:36.091372 20075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:36.092376 20069 server_base.cc:1034] running on GCE node
I20260806 09:02:36.093246 20069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:36.095023 20069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:36.096267 20069 hybrid_clock.cc:648] HybridClock initialized: now 1786006956096208 us; error 48 us; skew 500 ppm
I20260806 09:02:36.096665 20069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:36 dist-test-slave-jpj1 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006956, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:36.100054 20069 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:36.101748 20069 webserver.cc:460] Webserver started at http://127.0.0.1:43565/ using document root <none> and password file <none>
I20260806 09:02:36.102476 20069 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:36.102588 20069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:36.102887 20069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:36.106210 20069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "347463fa22224128b6ffc70c6146298f"
format_stamp: "Formatted at 2026-08-06 09:02:36 on dist-test-slave-jpj1"
server_key: "7a40756639c069af6660210c809144da"
server_key_iv: "fb2fedee5ec7a9eb2b05d7d3c6889fe3"
server_key_version: "encryptionkey@0"
I20260806 09:02:36.107016 20069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "347463fa22224128b6ffc70c6146298f"
format_stamp: "Formatted at 2026-08-06 09:02:36 on dist-test-slave-jpj1"
server_key: "7a40756639c069af6660210c809144da"
server_key_iv: "fb2fedee5ec7a9eb2b05d7d3c6889fe3"
server_key_version: "encryptionkey@0"
I20260806 09:02:36.112793 20069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 09:02:36.116920 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:36.118541 20069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:02:36.118739 20069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "347463fa22224128b6ffc70c6146298f"
format_stamp: "Formatted at 2026-08-06 09:02:36 on dist-test-slave-jpj1"
server_key: "7a40756639c069af6660210c809144da"
server_key_iv: "fb2fedee5ec7a9eb2b05d7d3c6889fe3"
server_key_version: "encryptionkey@0"
I20260806 09:02:36.118978 20069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:36.125344 20069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:36.129899 20069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:36.130198 20069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:36.148834 20069 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42715
I20260806 09:02:36.148919 20156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42715 every 8 connection(s)
I20260806 09:02:36.150427 20069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:36.152009 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20069
I20260806 09:02:36.152189 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:36.152650 22920 external_mini_cluster.cc:1428] Setting key 506a5f4c13ea43854c4a0b26aabb6ef0
I20260806 09:02:36.155117 20157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:36.162117 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:36.154575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55979 (local address 127.0.0.1:42715)
0806 09:02:36.155466 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:36.155482 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:36.155564 (+ 82us) server_negotiation.cc:407] Connection header received
0806 09:02:36.157916 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:36.157947 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:36.158467 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:36.158963 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:36.160013 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:36.160021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:36.160039 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:36.160057 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:36.160155 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:36.160188 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:36.160423 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:36.160572 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 09:02:36.160805 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":355,"threads_started":1}
I20260806 09:02:36.166250 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:36.169728 20157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f: Bootstrap starting.
I20260806 09:02:36.174165 20157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:36.174573 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:36.171758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34415 (local address 127.0.0.1:42715)
0806 09:02:36.171969 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:36.171975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:36.172295 (+ 320us) server_negotiation.cc:407] Connection header received
0806 09:02:36.172455 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:36.172460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:36.172525 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:36.172637 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:36.173794 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:36.173801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:36.173806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:36.173833 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:36.173890 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:36.173914 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:36.174057 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:36.174208 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 09:02:36.174278 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:02:36.175623 20157 log.cc:824] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:36.178931 20157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f: No bootstrap required, opened a new log
I20260806 09:02:36.186950 20157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260806 09:02:36.187494 20157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:36.187647 20157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347463fa22224128b6ffc70c6146298f, State: Initialized, Role: FOLLOWER
I20260806 09:02:36.188380 20157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260806 09:02:36.188683 20157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:36.188853 20157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:36.189028 20157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:36.191475 20157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260806 09:02:36.192241 20157 leader_election.cc:302] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347463fa22224128b6ffc70c6146298f; no voters:
I20260806 09:02:36.192955 20157 leader_election.cc:288] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:36.193086 20167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:36.193787 20167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [term 1 LEADER]: Becoming Leader. State: Replica: 347463fa22224128b6ffc70c6146298f, State: Running, Role: LEADER
I20260806 09:02:36.194484 20167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260806 09:02:36.195601 20157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:36.197788 20171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347463fa22224128b6ffc70c6146298f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } } }
I20260806 09:02:36.198302 20171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:36.203411 20172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347463fa22224128b6ffc70c6146298f. Latest consensus state: current_term: 1 leader_uuid: "347463fa22224128b6ffc70c6146298f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347463fa22224128b6ffc70c6146298f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } } }
I20260806 09:02:36.204020 20172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:36.212827 20179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:36.217850 20179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:36.227950 20179 catalog_manager.cc:1347] Generated new cluster ID: e5a7fbe2a10d4bcf829b6673e9d61533
I20260806 09:02:36.228103 20179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:36.261960 20179 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:36.262439 20179 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:36.269064 20179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347463fa22224128b6ffc70c6146298f: Generated new TSK 0
I20260806 09:02:36.269996 20179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:36.290391 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20069
W20260806 09:02:36.291646 20152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55979
W20260806 09:02:36.292397 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20069: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:36.292531 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20069
2026-08-06T09:02:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:36 dist-test-slave-jpj1 krb5kdc[20196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:36 dist-test-slave-jpj1 krb5kdc[20196](info): set up 2 sockets
Aug 06 09:02:36 dist-test-slave-jpj1 krb5kdc[20196](info): commencing operation
krb5kdc: starting...
W20260806 09:02:37.520704 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:37 dist-test-slave-jpj1 krb5kdc[20196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:37.854358 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:38.218889 20214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:38.219224 20214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:38.219349 20214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:38.226797 20214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:38.226920 20214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:38.226979 20214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:38.227067 20214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:38.227121 20214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:38.237564 20214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:38.239032 20214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:38.240475 20214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:38.253336 20225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:38.253300 20222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:38.253384 20221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:38.260089 20214 server_base.cc:1034] running on GCE node
I20260806 09:02:38.261059 20214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:38.262646 20214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:38.263882 20214 hybrid_clock.cc:648] HybridClock initialized: now 1786006958263817 us; error 54 us; skew 500 ppm
I20260806 09:02:38.264298 20214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:38 dist-test-slave-jpj1 krb5kdc[20196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:38.271710 20214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:38.273445 20214 webserver.cc:460] Webserver started at http://127.0.0.1:42433/ using document root <none> and password file <none>
I20260806 09:02:38.274195 20214 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:38.274327 20214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:38.274682 20214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:38.277998 20214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "075fda838ea84941a09d7c47a36270b3"
format_stamp: "Formatted at 2026-08-06 09:02:38 on dist-test-slave-jpj1"
server_key: "f173f4433f7415787a1acc6aa971a15e"
server_key_iv: "c400830c0df7e4a1fbd66763221655b9"
server_key_version: "encryptionkey@0"
I20260806 09:02:38.278798 20214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "075fda838ea84941a09d7c47a36270b3"
format_stamp: "Formatted at 2026-08-06 09:02:38 on dist-test-slave-jpj1"
server_key: "f173f4433f7415787a1acc6aa971a15e"
server_key_iv: "c400830c0df7e4a1fbd66763221655b9"
server_key_version: "encryptionkey@0"
I20260806 09:02:38.284265 20214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:02:38.288048 20234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:38.289515 20214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:02:38.289713 20214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "075fda838ea84941a09d7c47a36270b3"
format_stamp: "Formatted at 2026-08-06 09:02:38 on dist-test-slave-jpj1"
server_key: "f173f4433f7415787a1acc6aa971a15e"
server_key_iv: "c400830c0df7e4a1fbd66763221655b9"
server_key_version: "encryptionkey@0"
I20260806 09:02:38.289913 20214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:38.329670 20214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:38.334209 20214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:38.334496 20214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:38.353569 20214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42677
I20260806 09:02:38.353598 20302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42677 every 8 connection(s)
I20260806 09:02:38.354995 20214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:38.358949 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20214
I20260806 09:02:38.359074 20304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:38.359120 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:38.359624 22920 external_mini_cluster.cc:1428] Setting key db59de69155e3f525030e640835b8b74
I20260806 09:02:38.374341 20304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3: Bootstrap starting.
Aug 06 09:02:38 dist-test-slave-jpj1 krb5kdc[20196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:02:38.378355 20304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:38.379837 20304 log.cc:824] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:38.383459 20304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3: No bootstrap required, opened a new log
I20260806 09:02:38.384558 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:38.361647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:58335 (local address 127.0.0.1:42677)
0806 09:02:38.362549 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:38.362565 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:38.362635 (+ 70us) server_negotiation.cc:407] Connection header received
0806 09:02:38.364417 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:38.364450 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:38.365064 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:38.365557 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:38.366968 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:38.368999 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:38.370348 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:38.371468 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:38.371708 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:38.376118 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:38.376131 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:38.376151 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:38.376202 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:38.380118 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:38.381020 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:38.381028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:38.381041 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:38.381269 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:38.381822 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:38.381825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:38.381827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:38.382366 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:38.382610 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:38.383113 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 09:02:38.383479 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":363,"threads_started":1}
I20260806 09:02:38.388949 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:38.393756 20304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } }
I20260806 09:02:38.394301 20304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:38.394436 20304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075fda838ea84941a09d7c47a36270b3, State: Initialized, Role: FOLLOWER
I20260806 09:02:38.395138 20304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } }
I20260806 09:02:38.395395 20304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:38.395563 20304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:38.395773 20304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:38.398535 20304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } }
I20260806 09:02:38.399139 20304 leader_election.cc:302] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 075fda838ea84941a09d7c47a36270b3; no voters:
I20260806 09:02:38.399816 20304 leader_election.cc:288] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:38.399986 20319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:38.401158 20319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [term 1 LEADER]: Becoming Leader. State: Replica: 075fda838ea84941a09d7c47a36270b3, State: Running, Role: LEADER
I20260806 09:02:38.401727 20319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } }
I20260806 09:02:38.402408 20304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:38.404560 20323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 075fda838ea84941a09d7c47a36270b3. Latest consensus state: current_term: 1 leader_uuid: "075fda838ea84941a09d7c47a36270b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } } }
I20260806 09:02:38.404989 20323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:38.405608 20322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "075fda838ea84941a09d7c47a36270b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075fda838ea84941a09d7c47a36270b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42677 } } }
I20260806 09:02:38.405928 20322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:38.413599 20326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:38.413980 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:38.394715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:38231 (local address 127.0.0.1:42677)
0806 09:02:38.394945 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:38.394953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:38.395526 (+ 573us) server_negotiation.cc:407] Connection header received
0806 09:02:38.395745 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:38.395751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:38.395848 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:38.395966 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:38.397535 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:38.398595 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:38.400173 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:38.400880 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:38.401242 (+ 362us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:38.404968 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:38.404980 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:38.404986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:38.405033 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:38.409065 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:38.410271 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:38.410277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:38.410281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:38.410500 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:38.411144 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:38.411192 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:38.411196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:38.411741 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:38.413109 (+ 1368us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:38.413334 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 09:02:38.413539 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 09:02:38.419433 20326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:38.429281 20326 catalog_manager.cc:1347] Generated new cluster ID: 4370d52aeba442308d0a042e31bc27af
I20260806 09:02:38.429423 20326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:38.443138 20326 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:38.444533 20326 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:38.455176 20326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 075fda838ea84941a09d7c47a36270b3: Generated new TSK 0
I20260806 09:02:38.456102 20326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:38.466626 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20214
I20260806 09:02:38.467964 20295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:38.560307 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20214
W20260806 09:02:38.584790 20194 connection.cc:535] client connection to 127.0.0.1:42677 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42677): Connection reset by peer (error 104)
2026-08-06T09:02:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2268 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:38 dist-test-slave-jpj1 krb5kdc[20353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:38 dist-test-slave-jpj1 krb5kdc[20353](info): set up 2 sockets
Aug 06 09:02:38 dist-test-slave-jpj1 krb5kdc[20353](info): commencing operation
krb5kdc: starting...
W20260806 09:02:39.777560 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:39 dist-test-slave-jpj1 krb5kdc[20353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:40.115355 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:40.475222 20372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:40.475564 20372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:40.475708 20372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:40.483160 20372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:40.483292 20372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:40.483351 20372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:40.483443 20372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:40.483498 20372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:40.493810 20372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:40.495222 20372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:40.496640 20372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:40.508699 20377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:40.508756 20378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:40.508663 20380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:40.509883 20372 server_base.cc:1034] running on GCE node
I20260806 09:02:40.510756 20372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:40.512251 20372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:40.513473 20372 hybrid_clock.cc:648] HybridClock initialized: now 1786006960513425 us; error 34 us; skew 500 ppm
I20260806 09:02:40.513883 20372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:40 dist-test-slave-jpj1 krb5kdc[20353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:40.517259 20372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:40.518869 20372 webserver.cc:460] Webserver started at http://127.0.0.1:45783/ using document root <none> and password file <none>
I20260806 09:02:40.519599 20372 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:40.519769 20372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:40.520092 20372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:40.523211 20372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "72a79c4d1ee14124922dc90ed7c1896e"
format_stamp: "Formatted at 2026-08-06 09:02:40 on dist-test-slave-jpj1"
server_key: "d86b509144e72879aa935ff4ce12edf6"
server_key_iv: "111f38dc3e4ccb4568c5b5ba8def093d"
server_key_version: "encryptionkey@0"
I20260806 09:02:40.524044 20372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "72a79c4d1ee14124922dc90ed7c1896e"
format_stamp: "Formatted at 2026-08-06 09:02:40 on dist-test-slave-jpj1"
server_key: "d86b509144e72879aa935ff4ce12edf6"
server_key_iv: "111f38dc3e4ccb4568c5b5ba8def093d"
server_key_version: "encryptionkey@0"
I20260806 09:02:40.529531 20372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:02:40.533211 20392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:40.534624 20372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:02:40.534786 20372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "72a79c4d1ee14124922dc90ed7c1896e"
format_stamp: "Formatted at 2026-08-06 09:02:40 on dist-test-slave-jpj1"
server_key: "d86b509144e72879aa935ff4ce12edf6"
server_key_iv: "111f38dc3e4ccb4568c5b5ba8def093d"
server_key_version: "encryptionkey@0"
I20260806 09:02:40.534979 20372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:40.552598 20372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:40.557201 20372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:40.557476 20372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:40.576195 20372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44853
I20260806 09:02:40.576256 20464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44853 every 8 connection(s)
I20260806 09:02:40.577602 20372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:40.579198 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20372
I20260806 09:02:40.579420 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:40.580011 22920 external_mini_cluster.cc:1428] Setting key f2417abb6ecd025380b975dee438c7dc
I20260806 09:02:40.582386 20465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:40.594514 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:40.581896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:50247 (local address 127.0.0.1:44853)
0806 09:02:40.582691 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:40.582704 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:40.582750 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:02:40.584465 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:40.584489 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:40.584897 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:40.585276 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:40.585919 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:40.587322 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:40.589306 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:40.590092 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:40.590248 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:40.591779 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:40.591791 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:40.591808 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:40.592462 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:40.592552 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:40.592581 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:40.592795 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:40.593161 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 09:02:40.593401 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":338,"threads_started":1}
I20260806 09:02:40.595795 20465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e: Bootstrap starting.
I20260806 09:02:40.598708 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:40.599303 20465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:40.600421 20465 log.cc:824] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:40.604002 20465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e: No bootstrap required, opened a new log
I20260806 09:02:40.610219 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:40.603140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:35241 (local address 127.0.0.1:44853)
0806 09:02:40.603341 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:40.603348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:40.603728 (+ 380us) server_negotiation.cc:407] Connection header received
0806 09:02:40.603943 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:40.603947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:40.604015 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:40.604104 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:40.604748 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:40.605533 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:40.607415 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:40.607995 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:40.608125 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:40.609358 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:40.609364 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:40.609367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:40.609387 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:40.609453 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:40.609472 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:40.609571 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:40.609811 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 09:02:40.609880 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:02:40.613430 20465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } }
I20260806 09:02:40.613842 20465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:40.613957 20465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a79c4d1ee14124922dc90ed7c1896e, State: Initialized, Role: FOLLOWER
I20260806 09:02:40.614583 20465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } }
I20260806 09:02:40.614821 20465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:40.614954 20465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:40.615130 20465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:40.617488 20465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } }
I20260806 09:02:40.618147 20465 leader_election.cc:302] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a79c4d1ee14124922dc90ed7c1896e; no voters:
I20260806 09:02:40.618832 20465 leader_election.cc:288] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:40.618973 20476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:40.619709 20476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [term 1 LEADER]: Becoming Leader. State: Replica: 72a79c4d1ee14124922dc90ed7c1896e, State: Running, Role: LEADER
I20260806 09:02:40.620344 20476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } }
I20260806 09:02:40.621253 20465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:40.623844 20477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a79c4d1ee14124922dc90ed7c1896e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } } }
I20260806 09:02:40.623967 20478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a79c4d1ee14124922dc90ed7c1896e. Latest consensus state: current_term: 1 leader_uuid: "72a79c4d1ee14124922dc90ed7c1896e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a79c4d1ee14124922dc90ed7c1896e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44853 } } }
I20260806 09:02:40.624351 20477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:40.624356 20478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:40.626093 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:40.632036 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:40.641798 20484 catalog_manager.cc:1347] Generated new cluster ID: 3c247ce609074998ba7142c255315113
I20260806 09:02:40.641942 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:40.654675 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:40.656210 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:40.661659 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a79c4d1ee14124922dc90ed7c1896e: Generated new TSK 0
I20260806 09:02:40.662484 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:40.669067 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20372
W20260806 09:02:40.670341 20459 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.88:50247
W20260806 09:02:40.671232 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20372: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:40.671381 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20372
W20260806 09:02:40.693851 20349 connection.cc:535] client connection to 127.0.0.1:44853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44853): Connection reset by peer (error 104)
2026-08-06T09:02:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:40 dist-test-slave-jpj1 krb5kdc[20510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:40 dist-test-slave-jpj1 krb5kdc[20510](info): set up 2 sockets
Aug 06 09:02:40 dist-test-slave-jpj1 krb5kdc[20510](info): commencing operation
krb5kdc: starting...
W20260806 09:02:41.887045 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:42 dist-test-slave-jpj1 krb5kdc[20510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:42.233376 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:42.597554 20529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:42.597878 20529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:42.598013 20529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:42.605412 20529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:42.605535 20529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:42.605609 20529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:42.605690 20529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:42.605743 20529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:42.615942 20529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:42.617271 20529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:42.618702 20529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:42.630477 20535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:42.630477 20536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:42.630540 20538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:42.631978 20529 server_base.cc:1034] running on GCE node
I20260806 09:02:42.632880 20529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:42.634403 20529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:42.635632 20529 hybrid_clock.cc:648] HybridClock initialized: now 1786006962635583 us; error 46 us; skew 500 ppm
I20260806 09:02:42.636004 20529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:42 dist-test-slave-jpj1 krb5kdc[20510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:42.639268 20529 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:42.640861 20529 webserver.cc:460] Webserver started at http://127.0.0.1:38495/ using document root <none> and password file <none>
I20260806 09:02:42.641551 20529 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:42.641676 20529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:42.641993 20529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:42.645190 20529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "1a8bb78c0c8744f398f44ad471eb3e17"
format_stamp: "Formatted at 2026-08-06 09:02:42 on dist-test-slave-jpj1"
server_key: "8b543aab6ead8b05687fc8910c9e5364"
server_key_iv: "f62652193e406e7636e40973d466406b"
server_key_version: "encryptionkey@0"
I20260806 09:02:42.645998 20529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "1a8bb78c0c8744f398f44ad471eb3e17"
format_stamp: "Formatted at 2026-08-06 09:02:42 on dist-test-slave-jpj1"
server_key: "8b543aab6ead8b05687fc8910c9e5364"
server_key_iv: "f62652193e406e7636e40973d466406b"
server_key_version: "encryptionkey@0"
I20260806 09:02:42.651769 20529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:02:42.655709 20547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:42.657790 20529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 09:02:42.658051 20529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "1a8bb78c0c8744f398f44ad471eb3e17"
format_stamp: "Formatted at 2026-08-06 09:02:42 on dist-test-slave-jpj1"
server_key: "8b543aab6ead8b05687fc8910c9e5364"
server_key_iv: "f62652193e406e7636e40973d466406b"
server_key_version: "encryptionkey@0"
I20260806 09:02:42.658341 20529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:42.664738 20529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:42.669587 20529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:42.669875 20529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:42.688148 20529 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45135
I20260806 09:02:42.688205 20611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45135 every 8 connection(s)
I20260806 09:02:42.689564 20529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:42.693531 20612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:42.698119 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20529
I20260806 09:02:42.698323 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:42.698899 22920 external_mini_cluster.cc:1428] Setting key a17e10814487a12f4255e2bb26b4794e
I20260806 09:02:42.705796 20612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17: Bootstrap starting.
I20260806 09:02:42.707774 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:42.700966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:49481 (local address 127.0.0.1:45135)
0806 09:02:42.701808 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:42.701822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:42.701866 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:02:42.704091 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:42.704120 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:42.704536 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:42.704901 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:42.706066 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:42.706073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:42.706089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:42.706104 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:42.706175 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:42.706197 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:42.706404 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:42.706508 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 09:02:42.706707 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":382,"threads_started":1}
I20260806 09:02:42.709435 20612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:42.710683 20612 log.cc:824] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:42.711854 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:42.714259 20612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17: No bootstrap required, opened a new log
I20260806 09:02:42.720330 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:42.717432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:35711 (local address 127.0.0.1:45135)
0806 09:02:42.717597 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:42.717602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:42.718009 (+ 407us) server_negotiation.cc:407] Connection header received
0806 09:02:42.718192 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:42.718196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:42.718248 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:42.718342 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:42.719386 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:42.719392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:42.719395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:42.719416 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:42.719464 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:42.719481 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:42.719688 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:42.719919 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 09:02:42.720004 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 09:02:42.723852 20612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260806 09:02:42.724257 20612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:42.724365 20612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8bb78c0c8744f398f44ad471eb3e17, State: Initialized, Role: FOLLOWER
I20260806 09:02:42.725085 20612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260806 09:02:42.725361 20612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:42.725503 20612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:42.725687 20612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:42.728101 20612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260806 09:02:42.728627 20612 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8bb78c0c8744f398f44ad471eb3e17; no voters:
I20260806 09:02:42.729202 20612 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:42.729338 20625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:42.730031 20625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8bb78c0c8744f398f44ad471eb3e17, State: Running, Role: LEADER
I20260806 09:02:42.730633 20625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260806 09:02:42.731673 20612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:42.733660 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } } }
I20260806 09:02:42.734160 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:42.735792 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8bb78c0c8744f398f44ad471eb3e17. Latest consensus state: current_term: 1 leader_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8bb78c0c8744f398f44ad471eb3e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } } }
I20260806 09:02:42.736186 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:42.736794 20638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:42.741962 20638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:42.753772 20638 catalog_manager.cc:1347] Generated new cluster ID: efb42d78bb6d4c4b8b24717a5c892eae
I20260806 09:02:42.753914 20638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:42.788141 20638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:42.788717 20638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:42.796160 20638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8bb78c0c8744f398f44ad471eb3e17: Generated new TSK 0
I20260806 09:02:42.797083 20638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:42.805011 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20529
W20260806 09:02:42.806118 20607 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.88:49481
W20260806 09:02:42.806836 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20529: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:42.806963 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20529
2026-08-06T09:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:42 dist-test-slave-jpj1 krb5kdc[20654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:42 dist-test-slave-jpj1 krb5kdc[20654](info): set up 2 sockets
Aug 06 09:02:42 dist-test-slave-jpj1 krb5kdc[20654](info): commencing operation
krb5kdc: starting...
W20260806 09:02:44.048125 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.011s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:44 dist-test-slave-jpj1 krb5kdc[20654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:44.384843 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40169
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:44.734447 20671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:44.734758 20671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:44.734880 20671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:44.742215 20671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:44.742342 20671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:44.742388 20671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:44.742450 20671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:44.742493 20671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:44.752628 20671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40169
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:44.753901 20671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:44.755205 20671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:44.767138 20679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:44.767175 20680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:44.767092 20682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:44.768150 20671 server_base.cc:1034] running on GCE node
I20260806 09:02:44.768971 20671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:44.770442 20671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:44.771633 20671 hybrid_clock.cc:648] HybridClock initialized: now 1786006964771585 us; error 40 us; skew 500 ppm
I20260806 09:02:44.771989 20671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:44 dist-test-slave-jpj1 krb5kdc[20654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:44.778669 20671 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:44.780253 20671 webserver.cc:460] Webserver started at http://127.0.0.1:46333/ using document root <none> and password file <none>
I20260806 09:02:44.780953 20671 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:44.781090 20671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:44.781412 20671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:44.784622 20671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "be2c6d62258945988ec0c811749d1e0d"
format_stamp: "Formatted at 2026-08-06 09:02:44 on dist-test-slave-jpj1"
server_key: "2b0e90524da2178e3f82a4614661351c"
server_key_iv: "1b30ec9723882847f30695e0d9d34174"
server_key_version: "encryptionkey@0"
I20260806 09:02:44.785439 20671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "be2c6d62258945988ec0c811749d1e0d"
format_stamp: "Formatted at 2026-08-06 09:02:44 on dist-test-slave-jpj1"
server_key: "2b0e90524da2178e3f82a4614661351c"
server_key_iv: "1b30ec9723882847f30695e0d9d34174"
server_key_version: "encryptionkey@0"
I20260806 09:02:44.791328 20671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:02:44.795066 20690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:44.796603 20671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:02:44.796798 20671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "be2c6d62258945988ec0c811749d1e0d"
format_stamp: "Formatted at 2026-08-06 09:02:44 on dist-test-slave-jpj1"
server_key: "2b0e90524da2178e3f82a4614661351c"
server_key_iv: "1b30ec9723882847f30695e0d9d34174"
server_key_version: "encryptionkey@0"
I20260806 09:02:44.797014 20671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:44.818547 20671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:44.822822 20671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:44.823087 20671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:44.841074 20671 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40169
I20260806 09:02:44.841109 20758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40169 every 8 connection(s)
I20260806 09:02:44.842511 20671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:44.846488 20759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:44.847251 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20671
I20260806 09:02:44.847448 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:44.848063 22920 external_mini_cluster.cc:1428] Setting key 0124ba7867883da415a88e4b6c4b1f36
I20260806 09:02:44.860975 20759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d: Bootstrap starting.
Aug 06 09:02:44 dist-test-slave-jpj1 krb5kdc[20654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:02:44.865001 20759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:44.866253 20759 log.cc:824] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:44.869627 20759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d: No bootstrap required, opened a new log
I20260806 09:02:44.872606 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:44.850084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:34429 (local address 127.0.0.1:40169)
0806 09:02:44.850859 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:44.850870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:44.850916 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:02:44.852943 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:44.852973 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:44.853411 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:44.853760 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:44.855640 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:44.856969 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:44.858772 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:44.859481 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:44.859666 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:44.864653 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:44.864661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:44.864678 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:44.864722 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:44.867866 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:44.868906 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:44.868914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:44.868921 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:44.869116 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:44.869659 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:44.869663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:44.869665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:44.870246 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:44.870515 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:44.871084 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 09:02:44.871385 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":332,"threads_started":1}
I20260806 09:02:44.876797 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:44.879293 20759 raft_consensus.cc:348] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } }
I20260806 09:02:44.879817 20759 raft_consensus.cc:374] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:44.879932 20759 raft_consensus.cc:729] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be2c6d62258945988ec0c811749d1e0d, State: Initialized, Role: FOLLOWER
I20260806 09:02:44.880684 20759 consensus_queue.cc:260] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } }
I20260806 09:02:44.880998 20759 raft_consensus.cc:388] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:44.881145 20759 raft_consensus.cc:482] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:44.881356 20759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:44.884137 20759 raft_consensus.cc:504] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } }
I20260806 09:02:44.884720 20759 leader_election.cc:302] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be2c6d62258945988ec0c811749d1e0d; no voters:
I20260806 09:02:44.885452 20759 leader_election.cc:288] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:44.885555 20777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:44.886675 20777 raft_consensus.cc:686] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [term 1 LEADER]: Becoming Leader. State: Replica: be2c6d62258945988ec0c811749d1e0d, State: Running, Role: LEADER
I20260806 09:02:44.887312 20777 consensus_queue.cc:237] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } }
I20260806 09:02:44.888008 20759 sys_catalog.cc:564] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:44.890254 20779 sys_catalog.cc:455] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader be2c6d62258945988ec0c811749d1e0d. Latest consensus state: current_term: 1 leader_uuid: "be2c6d62258945988ec0c811749d1e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } } }
I20260806 09:02:44.890654 20779 sys_catalog.cc:458] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:44.891201 20778 sys_catalog.cc:455] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be2c6d62258945988ec0c811749d1e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be2c6d62258945988ec0c811749d1e0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40169 } } }
I20260806 09:02:44.891508 20778 sys_catalog.cc:458] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:44.901219 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:44.882078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:40729 (local address 127.0.0.1:40169)
0806 09:02:44.882271 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:44.882278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:44.882814 (+ 536us) server_negotiation.cc:407] Connection header received
0806 09:02:44.882897 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:44.882903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:44.882988 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:44.883109 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:44.884729 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:44.885951 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:44.887800 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:44.888548 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:44.888674 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:44.892310 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:44.892319 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:44.892324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:44.892370 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:44.896428 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:44.897657 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:44.897665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:44.897669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:44.897861 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:44.898470 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:44.898476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:44.898479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:44.899008 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:44.899309 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:44.900518 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0806 09:02:44.900798 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260806 09:02:44.911080 20782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:44.917174 20782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:44.926549 20782 catalog_manager.cc:1347] Generated new cluster ID: 17343448a2a94c46ae4675d8213fa052
I20260806 09:02:44.926679 20782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:44.942615 20782 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:44.944613 20782 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:44.955435 20782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be2c6d62258945988ec0c811749d1e0d: Generated new TSK 0
I20260806 09:02:44.956563 20782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:44.971200 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20671
I20260806 09:02:44.972780 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:45.065616 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20671
W20260806 09:02:45.089967 20652 connection.cc:535] client connection to 127.0.0.1:40169 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40169): Connection reset by peer (error 104)
2026-08-06T09:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:45 dist-test-slave-jpj1 krb5kdc[20807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:45 dist-test-slave-jpj1 krb5kdc[20807](info): set up 2 sockets
Aug 06 09:02:45 dist-test-slave-jpj1 krb5kdc[20807](info): commencing operation
krb5kdc: starting...
W20260806 09:02:46.276513 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:46 dist-test-slave-jpj1 krb5kdc[20807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:46.623289 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:46.985605 20821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:46.985931 20821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:46.986066 20821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:46.993690 20821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:46.993824 20821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:46.993901 20821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:46.993978 20821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:46.994032 20821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:47.004382 20821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:47.005805 20821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:47.007220 20821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:47.020045 20827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:47.020037 20831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:47.020048 20835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:47.020967 20821 server_base.cc:1034] running on GCE node
I20260806 09:02:47.021847 20821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:47.023456 20821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:47.024739 20821 hybrid_clock.cc:648] HybridClock initialized: now 1786006967024685 us; error 39 us; skew 500 ppm
I20260806 09:02:47.025159 20821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:47 dist-test-slave-jpj1 krb5kdc[20807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:47.028692 20821 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:47.030766 20821 webserver.cc:460] Webserver started at http://127.0.0.1:43869/ using document root <none> and password file <none>
I20260806 09:02:47.031594 20821 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:47.031751 20821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:47.032089 20821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:47.035509 20821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "fdc377c6e1734bed853fe9f8e61dc818"
format_stamp: "Formatted at 2026-08-06 09:02:47 on dist-test-slave-jpj1"
server_key: "146a32e7b517430d7a7871153c329301"
server_key_iv: "879242d2d0d6613ae4354a96f759d499"
server_key_version: "encryptionkey@0"
I20260806 09:02:47.036616 20821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "fdc377c6e1734bed853fe9f8e61dc818"
format_stamp: "Formatted at 2026-08-06 09:02:47 on dist-test-slave-jpj1"
server_key: "146a32e7b517430d7a7871153c329301"
server_key_iv: "879242d2d0d6613ae4354a96f759d499"
server_key_version: "encryptionkey@0"
I20260806 09:02:47.042876 20821 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 09:02:47.047154 20842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:47.048962 20821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:02:47.049247 20821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "fdc377c6e1734bed853fe9f8e61dc818"
format_stamp: "Formatted at 2026-08-06 09:02:47 on dist-test-slave-jpj1"
server_key: "146a32e7b517430d7a7871153c329301"
server_key_iv: "879242d2d0d6613ae4354a96f759d499"
server_key_version: "encryptionkey@0"
I20260806 09:02:47.049497 20821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:47.071169 20821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:47.076287 20821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:47.076601 20821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:47.097004 20821 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32935
I20260806 09:02:47.097083 20911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32935 every 8 connection(s)
I20260806 09:02:47.098496 20821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:47.102805 20913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:47.107637 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20821
I20260806 09:02:47.107862 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:47.108433 22920 external_mini_cluster.cc:1428] Setting key 3e4018cd9f3d692750525b3f1618b92b
I20260806 09:02:47.115407 20913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818: Bootstrap starting.
I20260806 09:02:47.119323 20913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:47.120723 20913 log.cc:824] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:47.123808 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:47.110526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:40771 (local address 127.0.0.1:32935)
0806 09:02:47.111344 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:47.111356 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:47.111401 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:02:47.113790 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:47.113821 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:47.114294 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:47.114648 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:47.115328 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:47.116806 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:47.118742 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:47.119587 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:47.119756 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:47.121053 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:47.121062 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:47.121080 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:47.121835 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:47.121940 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:47.121980 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:47.122174 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:47.122481 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 09:02:47.122709 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":352,"threads_started":1}
I20260806 09:02:47.124338 20913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818: No bootstrap required, opened a new log
I20260806 09:02:47.127952 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:47.134318 20913 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260806 09:02:47.134903 20913 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:47.135046 20913 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc377c6e1734bed853fe9f8e61dc818, State: Initialized, Role: FOLLOWER
I20260806 09:02:47.135785 20913 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260806 09:02:47.136042 20913 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:47.136180 20913 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:47.136392 20913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:47.139636 20913 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260806 09:02:47.140367 20913 leader_election.cc:302] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc377c6e1734bed853fe9f8e61dc818; no voters:
I20260806 09:02:47.140829 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:47.133557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:35641 (local address 127.0.0.1:32935)
0806 09:02:47.133718 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:47.133723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:47.134119 (+ 396us) server_negotiation.cc:407] Connection header received
0806 09:02:47.134339 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:47.134343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:47.134416 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:47.134499 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:47.135147 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:47.136116 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:47.137993 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:47.138526 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:47.138636 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:47.139981 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:47.139986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:47.139990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:47.140014 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:47.140061 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:47.140078 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:47.140181 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:47.140435 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 09:02:47.140496 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 09:02:47.141213 20913 leader_election.cc:288] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:47.141343 20924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:47.142545 20924 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [term 1 LEADER]: Becoming Leader. State: Replica: fdc377c6e1734bed853fe9f8e61dc818, State: Running, Role: LEADER
I20260806 09:02:47.143236 20924 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260806 09:02:47.143772 20913 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:47.146386 20927 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc377c6e1734bed853fe9f8e61dc818. Latest consensus state: current_term: 1 leader_uuid: "fdc377c6e1734bed853fe9f8e61dc818" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260806 09:02:47.146798 20927 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:47.147285 20926 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc377c6e1734bed853fe9f8e61dc818" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc377c6e1734bed853fe9f8e61dc818" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260806 09:02:47.147614 20926 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:47.148726 20933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:47.154312 20933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:47.164562 20933 catalog_manager.cc:1347] Generated new cluster ID: f760851364a044b98094a8051bfc4a76
I20260806 09:02:47.164683 20933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:47.173576 20933 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:47.174932 20933 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:47.183167 20933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc377c6e1734bed853fe9f8e61dc818: Generated new TSK 0
I20260806 09:02:47.184373 20933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:47.197435 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20821
W20260806 09:02:47.198730 20906 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.88:40771
W20260806 09:02:47.199573 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20821: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:47.199716 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20821
W20260806 09:02:47.221961 20805 connection.cc:535] client connection to 127.0.0.1:32935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32935): Connection reset by peer (error 104)
2026-08-06T09:02:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2132 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:47 dist-test-slave-jpj1 krb5kdc[20958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:47 dist-test-slave-jpj1 krb5kdc[20958](info): set up 2 sockets
Aug 06 09:02:47 dist-test-slave-jpj1 krb5kdc[20958](info): commencing operation
krb5kdc: starting...
W20260806 09:02:48.421159 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:48 dist-test-slave-jpj1 krb5kdc[20958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:02:48.763165 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:49.118584 20976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:49.118943 20976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:49.119072 20976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:49.126788 20976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:49.126935 20976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:49.126984 20976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:49.127072 20976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:49.127136 20976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:49.137881 20976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:49.139420 20976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:49.140981 20976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:49.154868 20981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:49.154893 20982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:49.155507 20976 server_base.cc:1034] running on GCE node
W20260806 09:02:49.155592 20984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:49.156565 20976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:49.158313 20976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:49.159595 20976 hybrid_clock.cc:648] HybridClock initialized: now 1786006969159480 us; error 83 us; skew 500 ppm
I20260806 09:02:49.159996 20976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:49 dist-test-slave-jpj1 krb5kdc[20958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:49.163405 20976 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:02:49.165172 20976 webserver.cc:460] Webserver started at http://127.0.0.1:34825/ using document root <none> and password file <none>
I20260806 09:02:49.165951 20976 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:49.166081 20976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:49.166414 20976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:49.169767 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7"
format_stamp: "Formatted at 2026-08-06 09:02:49 on dist-test-slave-jpj1"
server_key: "c38f3c4425bc0250e569c70a13bb12e7"
server_key_iv: "7e01a196333c4ca4490f64fb95f4819b"
server_key_version: "encryptionkey@0"
I20260806 09:02:49.170619 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7"
format_stamp: "Formatted at 2026-08-06 09:02:49 on dist-test-slave-jpj1"
server_key: "c38f3c4425bc0250e569c70a13bb12e7"
server_key_iv: "7e01a196333c4ca4490f64fb95f4819b"
server_key_version: "encryptionkey@0"
I20260806 09:02:49.176647 20976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 09:02:49.180583 20993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:49.182093 20976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:02:49.182299 20976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7"
format_stamp: "Formatted at 2026-08-06 09:02:49 on dist-test-slave-jpj1"
server_key: "c38f3c4425bc0250e569c70a13bb12e7"
server_key_iv: "7e01a196333c4ca4490f64fb95f4819b"
server_key_version: "encryptionkey@0"
I20260806 09:02:49.182521 20976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:49.188876 20976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:49.193606 20976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:49.193933 20976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:49.213044 20976 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38233
I20260806 09:02:49.213080 21068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38233 every 8 connection(s)
I20260806 09:02:49.214607 20976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:49.216131 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 20976
I20260806 09:02:49.216312 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:49.216756 22920 external_mini_cluster.cc:1428] Setting key e9a5166e0f96287acf43ed20399138cd
I20260806 09:02:49.219802 21069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:49.225679 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:49.218770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:52119 (local address 127.0.0.1:38233)
0806 09:02:49.219779 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:49.219800 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:49.220110 (+ 310us) server_negotiation.cc:407] Connection header received
0806 09:02:49.222000 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:49.222028 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:49.222404 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:49.222800 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:49.223955 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:49.223962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:49.223977 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:49.223993 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:49.224084 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:49.224105 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:49.224294 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:49.224403 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 09:02:49.224621 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":392,"threads_started":1}
I20260806 09:02:49.230197 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:49.232913 21069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7: Bootstrap starting.
I20260806 09:02:49.237082 21069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:49.237689 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:49.234820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.88:38459 (local address 127.0.0.1:38233)
0806 09:02:49.234992 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:49.234998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:49.235449 (+ 451us) server_negotiation.cc:407] Connection header received
0806 09:02:49.235660 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:49.235664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:49.235720 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:49.235805 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:49.236917 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:49.236922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:49.236925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:49.236944 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:49.236993 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:49.237010 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:49.237118 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:49.237297 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 09:02:49.237371 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 09:02:49.238517 21069 log.cc:824] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:49.241580 21069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7: No bootstrap required, opened a new log
I20260806 09:02:49.249389 21069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } }
I20260806 09:02:49.249883 21069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:49.249993 21069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b387ac1c25c4f70820ecc79c2ce3bf7, State: Initialized, Role: FOLLOWER
I20260806 09:02:49.250622 21069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } }
I20260806 09:02:49.250852 21069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:49.250989 21069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:49.251158 21069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:49.253500 21069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } }
I20260806 09:02:49.254165 21069 leader_election.cc:302] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b387ac1c25c4f70820ecc79c2ce3bf7; no voters:
I20260806 09:02:49.254837 21069 leader_election.cc:288] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:49.255003 21083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:49.255781 21083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [term 1 LEADER]: Becoming Leader. State: Replica: 3b387ac1c25c4f70820ecc79c2ce3bf7, State: Running, Role: LEADER
I20260806 09:02:49.256374 21083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } }
I20260806 09:02:49.257014 21069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:49.260257 21085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b387ac1c25c4f70820ecc79c2ce3bf7. Latest consensus state: current_term: 1 leader_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } } }
I20260806 09:02:49.260835 21085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:49.259972 21084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b387ac1c25c4f70820ecc79c2ce3bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38233 } } }
I20260806 09:02:49.261626 21084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:49.263350 21095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:49.268947 21095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:49.279301 21095 catalog_manager.cc:1347] Generated new cluster ID: 4e730a43b21a45c59bce5eff4444ab73
I20260806 09:02:49.279426 21095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:49.296445 21095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:49.296965 21095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:49.308111 21095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b387ac1c25c4f70820ecc79c2ce3bf7: Generated new TSK 0
I20260806 09:02:49.309068 21095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:49.315416 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20976
W20260806 09:02:49.317288 21063 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.88:52119
W20260806 09:02:49.318054 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 20976: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:49.318202 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 20976
2026-08-06T09:02:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2124 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:49 dist-test-slave-jpj1 krb5kdc[21110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:49 dist-test-slave-jpj1 krb5kdc[21110](info): set up 2 sockets
Aug 06 09:02:49 dist-test-slave-jpj1 krb5kdc[21110](info): commencing operation
krb5kdc: starting...
W20260806 09:02:50.542953 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:50 dist-test-slave-jpj1 krb5kdc[21110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:50.885030 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:38225
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:51.250814 21131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:51.251157 21131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:51.251256 21131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:51.258838 21131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:51.258981 21131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:51.259037 21131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:51.259101 21131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:51.259157 21131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:51.269769 21131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:38225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:51.271149 21131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:51.272553 21131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:51.285475 21138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:51.285395 21139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:51.285429 21142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:51.286427 21131 server_base.cc:1034] running on GCE node
I20260806 09:02:51.287354 21131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:51.289032 21131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:51.290262 21131 hybrid_clock.cc:648] HybridClock initialized: now 1786006971290206 us; error 55 us; skew 500 ppm
I20260806 09:02:51.290637 21131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:51 dist-test-slave-jpj1 krb5kdc[21110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006971, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:51.298369 21131 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:51.300309 21131 webserver.cc:460] Webserver started at http://127.22.98.62:36951/ using document root <none> and password file <none>
I20260806 09:02:51.301093 21131 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:51.301242 21131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:51.301570 21131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:51.304981 21131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "c797de203a734351a34ecd62603815bd"
format_stamp: "Formatted at 2026-08-06 09:02:51 on dist-test-slave-jpj1"
server_key: "b59d75aee4b3205c595f61896dd3fd72"
server_key_iv: "4fdf5382ed550bdb6ca0ef3c4cbe1ec0"
server_key_version: "encryptionkey@0"
I20260806 09:02:51.305864 21131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "c797de203a734351a34ecd62603815bd"
format_stamp: "Formatted at 2026-08-06 09:02:51 on dist-test-slave-jpj1"
server_key: "b59d75aee4b3205c595f61896dd3fd72"
server_key_iv: "4fdf5382ed550bdb6ca0ef3c4cbe1ec0"
server_key_version: "encryptionkey@0"
I20260806 09:02:51.311839 21131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 09:02:51.316152 21151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:51.317710 21131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:02:51.317952 21131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "c797de203a734351a34ecd62603815bd"
format_stamp: "Formatted at 2026-08-06 09:02:51 on dist-test-slave-jpj1"
server_key: "b59d75aee4b3205c595f61896dd3fd72"
server_key_iv: "4fdf5382ed550bdb6ca0ef3c4cbe1ec0"
server_key_version: "encryptionkey@0"
I20260806 09:02:51.318164 21131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:51.345412 21131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:51.350554 21131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:51.350871 21131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:51.370975 21131 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:38225
I20260806 09:02:51.371012 21223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:38225 every 8 connection(s)
I20260806 09:02:51.372483 21131 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:51.376751 21225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:51.380885 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21131
I20260806 09:02:51.381074 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:51.381532 22920 external_mini_cluster.cc:1428] Setting key 9fb75f84ce990a7673754ba347f9d758
I20260806 09:02:51.391625 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd: Bootstrap starting.
I20260806 09:02:51.395728 21225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:51.396854 21225 log.cc:824] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:02:51 dist-test-slave-jpj1 krb5kdc[21110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:51.400429 21225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd: No bootstrap required, opened a new log
I20260806 09:02:51.410763 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:51.383726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57425 (local address 127.22.98.62:38225)
0806 09:02:51.384624 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:51.384640 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:51.384698 (+ 58us) server_negotiation.cc:407] Connection header received
0806 09:02:51.386662 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:51.386701 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:51.387418 (+ 717us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:51.388027 (+ 609us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:51.389564 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:51.391701 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:51.393091 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:51.394208 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:51.394397 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:51.399137 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:51.399152 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:51.399174 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:51.399224 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:51.403270 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:51.404683 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:51.404695 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:51.404707 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:51.404965 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:51.406375 (+ 1410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:51.406379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:51.406382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:51.407169 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:51.407605 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:51.408443 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 09:02:51.409090 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":387,"threads_started":1}
I20260806 09:02:51.411741 21225 raft_consensus.cc:348] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } }
I20260806 09:02:51.412129 21225 raft_consensus.cc:374] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:51.412249 21225 raft_consensus.cc:729] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c797de203a734351a34ecd62603815bd, State: Initialized, Role: FOLLOWER
I20260806 09:02:51.412998 21225 consensus_queue.cc:260] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } }
I20260806 09:02:51.413314 21225 raft_consensus.cc:388] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:51.413475 21225 raft_consensus.cc:482] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:51.413647 21225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:51.415597 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:51.416869 21225 raft_consensus.cc:504] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } }
I20260806 09:02:51.417501 21225 leader_election.cc:302] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c797de203a734351a34ecd62603815bd; no voters:
I20260806 09:02:51.418270 21225 leader_election.cc:288] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:51.418382 21234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:51.419611 21234 raft_consensus.cc:686] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [term 1 LEADER]: Becoming Leader. State: Replica: c797de203a734351a34ecd62603815bd, State: Running, Role: LEADER
I20260806 09:02:51.420245 21234 consensus_queue.cc:237] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } }
I20260806 09:02:51.420807 21225 sys_catalog.cc:564] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:51.422662 21237 sys_catalog.cc:455] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c797de203a734351a34ecd62603815bd. Latest consensus state: current_term: 1 leader_uuid: "c797de203a734351a34ecd62603815bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } } }
I20260806 09:02:51.423074 21237 sys_catalog.cc:458] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:51.423969 21235 sys_catalog.cc:455] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c797de203a734351a34ecd62603815bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c797de203a734351a34ecd62603815bd" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 38225 } } }
I20260806 09:02:51.424306 21235 sys_catalog.cc:458] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:51.424896 21241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:51.443228 21241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:51.454694 21241 catalog_manager.cc:1347] Generated new cluster ID: 439a596d27ee4f86a17b4c7b01800169
I20260806 09:02:51.454830 21241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:51.461787 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:51.440627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54367 (local address 127.22.98.62:38225)
0806 09:02:51.440848 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:51.440854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:51.445106 (+ 4252us) server_negotiation.cc:407] Connection header received
0806 09:02:51.445310 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:51.445317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:51.445418 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:51.445528 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:51.447878 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:51.449148 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:51.451068 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:51.451818 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:51.451930 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:51.455675 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:51.455681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:51.455684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:51.455771 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:51.458631 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:51.459522 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:51.459526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:51.459554 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:51.459732 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:51.460195 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:51.460198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:51.460201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:51.460502 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:51.460754 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:51.461183 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 09:02:51.461374 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:02:51.474191 21241 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:51.475677 21241 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:51.483902 21241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c797de203a734351a34ecd62603815bd: Generated new TSK 0
I20260806 09:02:51.484805 21241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:51.494689 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21131
I20260806 09:02:51.496052 21218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:51.589766 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21131
W20260806 09:02:51.613915 21108 connection.cc:535] client connection to 127.22.98.62:38225 recv error: Network error: failed to read from TLS socket (remote: 127.22.98.62:38225): Connection reset by peer (error 104)
2026-08-06T09:02:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:51 dist-test-slave-jpj1 krb5kdc[21265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:51 dist-test-slave-jpj1 krb5kdc[21265](info): set up 2 sockets
Aug 06 09:02:51 dist-test-slave-jpj1 krb5kdc[21265](info): commencing operation
krb5kdc: starting...
W20260806 09:02:52.798959 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.011s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:53 dist-test-slave-jpj1 krb5kdc[21265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:53.149026 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39465
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:45227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:53.510521 21281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:53.510856 21281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:53.511001 21281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:53.518391 21281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:53.518538 21281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:53.518623 21281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:53.518713 21281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:53.518772 21281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:53.529016 21281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:45227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:39465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:53.530453 21281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:53.531939 21281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:53.544202 21286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:53.544293 21287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:53.544173 21290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:53.545120 21281 server_base.cc:1034] running on GCE node
I20260806 09:02:53.545924 21281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:53.547343 21281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:53.548548 21281 hybrid_clock.cc:648] HybridClock initialized: now 1786006973548484 us; error 55 us; skew 500 ppm
I20260806 09:02:53.548902 21281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:53 dist-test-slave-jpj1 krb5kdc[21265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006973, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:53.552205 21281 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:53.553753 21281 webserver.cc:460] Webserver started at http://127.22.98.62:35757/ using document root <none> and password file <none>
I20260806 09:02:53.554466 21281 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:53.554593 21281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:53.554915 21281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:53.558045 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "8ee4ecdb73b34b4f813893a47a7d89cf"
format_stamp: "Formatted at 2026-08-06 09:02:53 on dist-test-slave-jpj1"
server_key: "b56f8fa6f5ff0a4a156d3e85628cb9c9"
server_key_iv: "b793bfe7c56ebecbf156d78aa12e6458"
server_key_version: "encryptionkey@0"
I20260806 09:02:53.558822 21281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "8ee4ecdb73b34b4f813893a47a7d89cf"
format_stamp: "Formatted at 2026-08-06 09:02:53 on dist-test-slave-jpj1"
server_key: "b56f8fa6f5ff0a4a156d3e85628cb9c9"
server_key_iv: "b793bfe7c56ebecbf156d78aa12e6458"
server_key_version: "encryptionkey@0"
I20260806 09:02:53.564218 21281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 09:02:53.567951 21301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:53.569468 21281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:02:53.569661 21281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "8ee4ecdb73b34b4f813893a47a7d89cf"
format_stamp: "Formatted at 2026-08-06 09:02:53 on dist-test-slave-jpj1"
server_key: "b56f8fa6f5ff0a4a156d3e85628cb9c9"
server_key_iv: "b793bfe7c56ebecbf156d78aa12e6458"
server_key_version: "encryptionkey@0"
I20260806 09:02:53.569886 21281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:53.580857 21281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:53.585000 21281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:53.585291 21281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:53.603590 21281 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:39465
I20260806 09:02:53.603688 21371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:39465 every 8 connection(s)
I20260806 09:02:53.605058 21281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:53.609006 21372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:53.615118 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21281
I20260806 09:02:53.615324 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:53.615897 22920 external_mini_cluster.cc:1428] Setting key 9f45a58cdfd520603f4714af48a693e3
I20260806 09:02:53.620702 21372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf: Bootstrap starting.
I20260806 09:02:53.624529 21372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:53.626015 21372 log.cc:824] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:53.629381 21372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf: No bootstrap required, opened a new log
I20260806 09:02:53.630192 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:53.617896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60369 (local address 127.22.98.62:39465)
0806 09:02:53.618767 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:53.618779 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:53.618824 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:02:53.620605 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:53.620630 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:53.621064 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:53.621437 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:53.622156 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:53.623622 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:53.625459 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:53.626249 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:53.626409 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:53.627761 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:53.627769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:53.627813 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:53.628370 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:53.628448 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:53.628476 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:53.628599 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:53.628901 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 09:02:53.629106 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":396,"threads_started":1}
I20260806 09:02:53.634321 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:53.637869 21372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } }
I20260806 09:02:53.638288 21372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:53.638399 21372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee4ecdb73b34b4f813893a47a7d89cf, State: Initialized, Role: FOLLOWER
I20260806 09:02:53.639147 21372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } }
I20260806 09:02:53.639420 21372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:53.639592 21372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:53.639796 21372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:53.642925 21372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } }
I20260806 09:02:53.643615 21372 leader_election.cc:302] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ee4ecdb73b34b4f813893a47a7d89cf; no voters:
I20260806 09:02:53.644362 21372 leader_election.cc:288] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:53.644491 21389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:53.645511 21389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [term 1 LEADER]: Becoming Leader. State: Replica: 8ee4ecdb73b34b4f813893a47a7d89cf, State: Running, Role: LEADER
I20260806 09:02:53.646092 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:53.638903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41815 (local address 127.22.98.62:39465)
0806 09:02:53.639096 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:53.639103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:53.639683 (+ 580us) server_negotiation.cc:407] Connection header received
0806 09:02:53.639908 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:53.639913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:53.640005 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:53.640132 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:53.640623 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:53.641800 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:53.643240 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:53.643837 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:53.643970 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:53.645243 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:53.645252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:53.645256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:53.645283 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:53.645347 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:53.645365 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:53.645462 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:53.645731 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 09:02:53.645791 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:02:53.646310 21389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } }
I20260806 09:02:53.648717 21389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } } }
I20260806 09:02:53.649142 21389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:53.649472 21390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ee4ecdb73b34b4f813893a47a7d89cf. Latest consensus state: current_term: 1 leader_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee4ecdb73b34b4f813893a47a7d89cf" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39465 } } }
I20260806 09:02:53.649930 21390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:53.653199 21372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:53.653450 21393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:53.659468 21393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:53.670043 21393 catalog_manager.cc:1347] Generated new cluster ID: e4af0608e503432c9cdab1174564b367
I20260806 09:02:53.670187 21393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:53.705073 21393 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:53.706526 21393 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:53.722201 21393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ee4ecdb73b34b4f813893a47a7d89cf: Generated new TSK 0
I20260806 09:02:53.723215 21393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:53.741539 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21281
W20260806 09:02:53.742753 21367 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60369
W20260806 09:02:53.743526 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21281: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:53.743707 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21281
W20260806 09:02:53.767899 21263 connection.cc:535] client connection to 127.22.98.62:39465 recv error: Network error: failed to read from TLS socket (remote: 127.22.98.62:39465): Connection reset by peer (error 104)
2026-08-06T09:02:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:53 dist-test-slave-jpj1 krb5kdc[21414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:53 dist-test-slave-jpj1 krb5kdc[21414](info): set up 2 sockets
Aug 06 09:02:53 dist-test-slave-jpj1 krb5kdc[21414](info): commencing operation
krb5kdc: starting...
W20260806 09:02:54.953577 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:55 dist-test-slave-jpj1 krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:55.278698 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39983
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:39983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:55.629549 21428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:55.629870 21428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:55.629994 21428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:55.637351 21428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:55.637470 21428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:55.637583 21428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:55.637679 21428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:55.637748 21428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:55.647782 21428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:39983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:39983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:55.649137 21428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:55.650484 21428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:55.662408 21436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:55.662478 21433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:55.662478 21434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:55.662899 21428 server_base.cc:1034] running on GCE node
I20260806 09:02:55.664011 21428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:55.665444 21428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:55.666661 21428 hybrid_clock.cc:648] HybridClock initialized: now 1786006975666612 us; error 33 us; skew 500 ppm
I20260806 09:02:55.667060 21428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:55 dist-test-slave-jpj1 krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006975, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:55.670384 21428 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:55.671937 21428 webserver.cc:460] Webserver started at http://127.22.98.62:35361/ using document root <none> and password file <none>
I20260806 09:02:55.672616 21428 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:55.672736 21428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:55.673072 21428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:55.676187 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "b54d580a15434eb590e87974c2ba6416"
format_stamp: "Formatted at 2026-08-06 09:02:55 on dist-test-slave-jpj1"
server_key: "b50a61d1ca208a1644751563387d9553"
server_key_iv: "1e3225769a9f3b05f7d949f80d41c9bf"
server_key_version: "encryptionkey@0"
I20260806 09:02:55.676968 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "b54d580a15434eb590e87974c2ba6416"
format_stamp: "Formatted at 2026-08-06 09:02:55 on dist-test-slave-jpj1"
server_key: "b50a61d1ca208a1644751563387d9553"
server_key_iv: "1e3225769a9f3b05f7d949f80d41c9bf"
server_key_version: "encryptionkey@0"
I20260806 09:02:55.682235 21428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:02:55.686004 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:55.687417 21428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:02:55.687624 21428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "b54d580a15434eb590e87974c2ba6416"
format_stamp: "Formatted at 2026-08-06 09:02:55 on dist-test-slave-jpj1"
server_key: "b50a61d1ca208a1644751563387d9553"
server_key_iv: "1e3225769a9f3b05f7d949f80d41c9bf"
server_key_version: "encryptionkey@0"
I20260806 09:02:55.687870 21428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:55.693734 21428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:55.698035 21428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:55.698326 21428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:55.717907 21428 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:39983
I20260806 09:02:55.717965 21516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:39983 every 8 connection(s)
I20260806 09:02:55.719378 21428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:55.720410 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21428
I20260806 09:02:55.720616 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:55.721168 22920 external_mini_cluster.cc:1428] Setting key 9f204bfbe00aa03c6e5f3f491257bf79
I20260806 09:02:55.724323 21517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:55.728930 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:55.723054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52407 (local address 127.22.98.62:39983)
0806 09:02:55.723905 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:55.723918 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:55.723962 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:02:55.725629 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:55.725652 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:55.725973 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:55.726323 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:55.727323 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:55.727333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:55.727350 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:55.727372 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:55.727442 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:55.727463 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:55.727688 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:55.727844 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 09:02:55.728021 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":352,"threads_started":1}
I20260806 09:02:55.732896 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:55.736083 21517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416: Bootstrap starting.
I20260806 09:02:55.739408 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:55.737152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37325 (local address 127.22.98.62:39983)
0806 09:02:55.737322 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:55.737329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:55.737637 (+ 308us) server_negotiation.cc:407] Connection header received
0806 09:02:55.737763 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:55.737769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:55.737840 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:55.737953 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:55.738811 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:55.738818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:55.738823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:02:55.738842 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:55.738897 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:02:55.738918 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:55.739035 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:55.739080 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 09:02:55.739143 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:02:55.739800 21517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:55.740885 21517 log.cc:824] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:55.743798 21517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416: No bootstrap required, opened a new log
I20260806 09:02:55.750993 21517 raft_consensus.cc:348] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } }
I20260806 09:02:55.751368 21517 raft_consensus.cc:374] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:55.751469 21517 raft_consensus.cc:729] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54d580a15434eb590e87974c2ba6416, State: Initialized, Role: FOLLOWER
I20260806 09:02:55.752144 21517 consensus_queue.cc:260] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } }
I20260806 09:02:55.752373 21517 raft_consensus.cc:388] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:55.752511 21517 raft_consensus.cc:482] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:55.752684 21517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:55.754863 21517 raft_consensus.cc:504] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } }
I20260806 09:02:55.755393 21517 leader_election.cc:302] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b54d580a15434eb590e87974c2ba6416; no voters:
I20260806 09:02:55.756023 21517 leader_election.cc:288] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:55.756147 21530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:55.756769 21530 raft_consensus.cc:686] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [term 1 LEADER]: Becoming Leader. State: Replica: b54d580a15434eb590e87974c2ba6416, State: Running, Role: LEADER
I20260806 09:02:55.757337 21530 consensus_queue.cc:237] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } }
I20260806 09:02:55.758080 21517 sys_catalog.cc:564] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:55.760352 21532 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b54d580a15434eb590e87974c2ba6416. Latest consensus state: current_term: 1 leader_uuid: "b54d580a15434eb590e87974c2ba6416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } } }
I20260806 09:02:55.760859 21532 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:55.761447 21531 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b54d580a15434eb590e87974c2ba6416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54d580a15434eb590e87974c2ba6416" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 39983 } } }
I20260806 09:02:55.761744 21531 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:55.761976 21540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:55.768100 21540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:55.777832 21540 catalog_manager.cc:1347] Generated new cluster ID: fc906430cd804f18949fd1ba14d2a354
I20260806 09:02:55.777968 21540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:55.797875 21540 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:55.798365 21540 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:55.807814 21540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b54d580a15434eb590e87974c2ba6416: Generated new TSK 0
I20260806 09:02:55.808728 21540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:55.814466 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21428
W20260806 09:02:55.815707 21510 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52407
W20260806 09:02:55.816437 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21428: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:02:55.816601 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21428
2026-08-06T09:02:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2073 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:55 dist-test-slave-jpj1 krb5kdc[21559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:55 dist-test-slave-jpj1 krb5kdc[21559](info): set up 2 sockets
Aug 06 09:02:55 dist-test-slave-jpj1 krb5kdc[21559](info): commencing operation
krb5kdc: starting...
W20260806 09:02:57.026399 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:57 dist-test-slave-jpj1 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:57.356670 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36657
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:42597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:36657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:57.718528 21574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:57.718850 21574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:57.718956 21574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:57.726418 21574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:57.726547 21574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:57.726603 21574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:57.726665 21574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:57.726717 21574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:57.736774 21574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:42597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:36657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:36657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:57.738102 21574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:57.739483 21574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:57.751343 21581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:57.751343 21582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:57.751385 21585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:57.752462 21574 server_base.cc:1034] running on GCE node
I20260806 09:02:57.753352 21574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:57.754834 21574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:57.756040 21574 hybrid_clock.cc:648] HybridClock initialized: now 1786006977755990 us; error 48 us; skew 500 ppm
I20260806 09:02:57.756395 21574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:57 dist-test-slave-jpj1 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006977, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:02:57.764232 21574 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:02:57.766068 21574 webserver.cc:460] Webserver started at http://127.22.98.62:33641/ using document root <none> and password file <none>
I20260806 09:02:57.766927 21574 fs_manager.cc:359] Metadata directory not provided
I20260806 09:02:57.767059 21574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:02:57.767377 21574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:02:57.770650 21574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "c4a79abb7b2745868cab77dc425108e3"
format_stamp: "Formatted at 2026-08-06 09:02:57 on dist-test-slave-jpj1"
server_key: "9892cfe298b66aeebda4ebb200b00107"
server_key_iv: "f25c88f1fca13feaeb1ecac63e20d5ce"
server_key_version: "encryptionkey@0"
I20260806 09:02:57.771416 21574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "c4a79abb7b2745868cab77dc425108e3"
format_stamp: "Formatted at 2026-08-06 09:02:57 on dist-test-slave-jpj1"
server_key: "9892cfe298b66aeebda4ebb200b00107"
server_key_iv: "f25c88f1fca13feaeb1ecac63e20d5ce"
server_key_version: "encryptionkey@0"
I20260806 09:02:57.776886 21574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 09:02:57.780555 21592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:02:57.782042 21574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:02:57.782241 21574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "c4a79abb7b2745868cab77dc425108e3"
format_stamp: "Formatted at 2026-08-06 09:02:57 on dist-test-slave-jpj1"
server_key: "9892cfe298b66aeebda4ebb200b00107"
server_key_iv: "f25c88f1fca13feaeb1ecac63e20d5ce"
server_key_version: "encryptionkey@0"
I20260806 09:02:57.782444 21574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:02:57.812274 21574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:02:57.816944 21574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:02:57.817238 21574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:02:57.835551 21574 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:36657
I20260806 09:02:57.835595 21670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:36657 every 8 connection(s)
I20260806 09:02:57.836987 21574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:02:57.840583 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21574
I20260806 09:02:57.840749 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:02:57.840862 21671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:02:57.841339 22920 external_mini_cluster.cc:1428] Setting key b2b8e5c8b29c40c4978ec1982a9a2b2d
I20260806 09:02:57.855660 21671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3: Bootstrap starting.
Aug 06 09:02:57 dist-test-slave-jpj1 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:02:57.859771 21671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3: Neither blocks nor log segments found. Creating new log.
I20260806 09:02:57.861251 21671 log.cc:824] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:02:57.865091 21671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3: No bootstrap required, opened a new log
I20260806 09:02:57.866726 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:57.843188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50921 (local address 127.22.98.62:36657)
0806 09:02:57.844097 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:57.844112 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:57.844167 (+ 55us) server_negotiation.cc:407] Connection header received
0806 09:02:57.845946 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:57.845985 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:57.846534 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:57.847033 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:57.848504 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:57.850469 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:57.851841 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:57.852932 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:57.853127 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:57.857424 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:57.857437 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:57.857460 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:57.857528 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:57.861682 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:57.862750 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:57.862758 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:57.862767 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:57.863005 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:57.863414 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:57.863419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:57.863422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:57.864198 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:57.864507 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:57.864853 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 09:02:57.865301 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":381,"threads_started":1}
I20260806 09:02:57.870854 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:02:57.874030 21671 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } }
I20260806 09:02:57.874480 21671 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:02:57.874577 21671 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a79abb7b2745868cab77dc425108e3, State: Initialized, Role: FOLLOWER
I20260806 09:02:57.875241 21671 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } }
I20260806 09:02:57.875567 21671 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:02:57.875721 21671 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:02:57.875918 21671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:02:57.878863 21671 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } }
I20260806 09:02:57.879487 21671 leader_election.cc:302] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4a79abb7b2745868cab77dc425108e3; no voters:
I20260806 09:02:57.880162 21671 leader_election.cc:288] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:02:57.880270 21688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:02:57.881856 21688 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [term 1 LEADER]: Becoming Leader. State: Replica: c4a79abb7b2745868cab77dc425108e3, State: Running, Role: LEADER
I20260806 09:02:57.882483 21671 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:02:57.882644 21688 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } }
I20260806 09:02:57.885301 21690 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4a79abb7b2745868cab77dc425108e3. Latest consensus state: current_term: 1 leader_uuid: "c4a79abb7b2745868cab77dc425108e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } } }
I20260806 09:02:57.885681 21690 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:57.886300 21689 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4a79abb7b2745868cab77dc425108e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a79abb7b2745868cab77dc425108e3" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 36657 } } }
I20260806 09:02:57.886595 21689 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:02:57.889052 21694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:02:57.892215 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:02:57.875078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40667 (local address 127.22.98.62:36657)
0806 09:02:57.875290 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 09:02:57.875298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:02:57.875859 (+ 561us) server_negotiation.cc:407] Connection header received
0806 09:02:57.876052 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:02:57.876058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:02:57.876134 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:02:57.876228 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:02:57.878053 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:57.878763 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:57.880635 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:02:57.881397 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:02:57.881533 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:02:57.884498 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:02:57.884509 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:02:57.884515 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:02:57.884558 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:02:57.888528 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:57.889447 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:57.889454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:57.889457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:57.889643 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:02:57.890142 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:02:57.890147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:02:57.890151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:02:57.890587 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:02:57.890890 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 09:02:57.891504 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 09:02:57.891810 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:02:57.895629 21694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:02:57.905524 21694 catalog_manager.cc:1347] Generated new cluster ID: 1f85c3ad19ad4da1b1392cf08739935d
I20260806 09:02:57.905639 21694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:02:57.923036 21694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:02:57.925021 21694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:02:57.933178 21694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4a79abb7b2745868cab77dc425108e3: Generated new TSK 0
I20260806 09:02:57.934304 21694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:02:57.951334 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21574
I20260806 09:02:57.952802 21665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:02:58.044560 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21574
2026-08-06T09:02:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2228 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:02:58 dist-test-slave-jpj1 krb5kdc[21717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:02:58 dist-test-slave-jpj1 krb5kdc[21717](info): set up 2 sockets
Aug 06 09:02:58 dist-test-slave-jpj1 krb5kdc[21717](info): commencing operation
krb5kdc: starting...
W20260806 09:02:59.270359 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:02:59 dist-test-slave-jpj1 krb5kdc[21717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:02:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:02:59.610656 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34849
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:33989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:34849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:02:59.961869 21733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:02:59.962172 21733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:02:59.962299 21733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:02:59.969565 21733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:02:59.969687 21733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:02:59.969760 21733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:02:59.969843 21733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:02:59.969900 21733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:02:59.979686 21733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:33989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:34849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:34849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:02:59.981006 21733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:02:59.982381 21733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:02:59.994438 21741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:59.994415 21742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:02:59.994369 21744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:02:59.995611 21733 server_base.cc:1034] running on GCE node
I20260806 09:02:59.996464 21733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:02:59.997932 21733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:02:59.999137 21733 hybrid_clock.cc:648] HybridClock initialized: now 1786006979999082 us; error 48 us; skew 500 ppm
I20260806 09:02:59.999516 21733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:02:59 dist-test-slave-jpj1 krb5kdc[21717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006979, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:00.003345 21733 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:03:00.005003 21733 webserver.cc:460] Webserver started at http://127.22.98.62:40305/ using document root <none> and password file <none>
I20260806 09:03:00.005745 21733 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:00.005937 21733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:00.006336 21733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:00.009699 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "b720c3203206420084a5aef3f46413e6"
format_stamp: "Formatted at 2026-08-06 09:03:00 on dist-test-slave-jpj1"
server_key: "941635fc89df22d5f3f20799f0495741"
server_key_iv: "fbe4bd908f91c5679d3e3603211b4705"
server_key_version: "encryptionkey@0"
I20260806 09:03:00.010555 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "b720c3203206420084a5aef3f46413e6"
format_stamp: "Formatted at 2026-08-06 09:03:00 on dist-test-slave-jpj1"
server_key: "941635fc89df22d5f3f20799f0495741"
server_key_iv: "fbe4bd908f91c5679d3e3603211b4705"
server_key_version: "encryptionkey@0"
I20260806 09:03:00.016328 21733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:03:00.020128 21753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:00.021584 21733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:03:00.021826 21733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "b720c3203206420084a5aef3f46413e6"
format_stamp: "Formatted at 2026-08-06 09:03:00 on dist-test-slave-jpj1"
server_key: "941635fc89df22d5f3f20799f0495741"
server_key_iv: "fbe4bd908f91c5679d3e3603211b4705"
server_key_version: "encryptionkey@0"
I20260806 09:03:00.022022 21733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:00.036335 21733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:00.040597 21733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:00.040850 21733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:00.058924 21733 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:34849
I20260806 09:03:00.058961 21828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:34849 every 8 connection(s)
I20260806 09:03:00.060356 21733 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:03:00.064016 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21733
I20260806 09:03:00.064263 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:03:00.064630 21829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:00.064891 22920 external_mini_cluster.cc:1428] Setting key be3c1fd6a3f508ffd9d82db3da637d6b
I20260806 09:03:00.079746 21829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6: Bootstrap starting.
I20260806 09:03:00.079787 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:00.066976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36433 (local address 127.22.98.62:34849)
0806 09:03:00.067874 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:00.067886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:00.067935 (+ 49us) server_negotiation.cc:407] Connection header received
0806 09:03:00.069880 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:00.069908 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:00.070363 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:00.070767 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:00.071456 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:00.072949 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:00.074814 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:00.075637 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:00.075811 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:00.077203 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:00.077216 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:00.077233 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:03:00.077890 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:00.077987 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:03:00.078015 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:00.078104 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:00.078351 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 09:03:00.078640 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":358,"threads_started":1}
I20260806 09:03:00.083110 21829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:00.083634 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:03:00.084409 21829 log.cc:824] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:00.087965 21829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6: No bootstrap required, opened a new log
I20260806 09:03:00.095713 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:00.088485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55527 (local address 127.22.98.62:34849)
0806 09:03:00.088696 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:00.088703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:00.089299 (+ 596us) server_negotiation.cc:407] Connection header received
0806 09:03:00.089434 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:00.089440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:00.089520 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:00.089659 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:00.090104 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:00.091223 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:00.092566 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:00.093304 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:00.093440 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:00.094825 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:00.094832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:00.094837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:03:00.094862 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:00.094911 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:03:00.094936 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:00.095036 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:00.095219 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 09:03:00.095311 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:03:00.098002 21829 raft_consensus.cc:348] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } }
I20260806 09:03:00.098371 21829 raft_consensus.cc:374] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:03:00.098445 21829 raft_consensus.cc:729] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b720c3203206420084a5aef3f46413e6, State: Initialized, Role: FOLLOWER
I20260806 09:03:00.099014 21829 consensus_queue.cc:260] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } }
I20260806 09:03:00.099279 21829 raft_consensus.cc:388] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:03:00.099445 21829 raft_consensus.cc:482] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:03:00.099681 21829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:00.101922 21829 raft_consensus.cc:504] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } }
I20260806 09:03:00.102458 21829 leader_election.cc:302] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b720c3203206420084a5aef3f46413e6; no voters:
I20260806 09:03:00.103022 21829 leader_election.cc:288] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:03:00.103158 21842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:03:00.103809 21842 raft_consensus.cc:686] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [term 1 LEADER]: Becoming Leader. State: Replica: b720c3203206420084a5aef3f46413e6, State: Running, Role: LEADER
I20260806 09:03:00.104468 21842 consensus_queue.cc:237] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } }
I20260806 09:03:00.105319 21829 sys_catalog.cc:564] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:00.107398 21845 sys_catalog.cc:455] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b720c3203206420084a5aef3f46413e6. Latest consensus state: current_term: 1 leader_uuid: "b720c3203206420084a5aef3f46413e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } } }
I20260806 09:03:00.107831 21845 sys_catalog.cc:458] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:00.107726 21843 sys_catalog.cc:455] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b720c3203206420084a5aef3f46413e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b720c3203206420084a5aef3f46413e6" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 34849 } } }
I20260806 09:03:00.108021 21843 sys_catalog.cc:458] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6 [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:00.109949 21852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:03:00.115080 21852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:03:00.124963 21852 catalog_manager.cc:1347] Generated new cluster ID: 39dbf8806a7a49238d3b9e2110a64c65
I20260806 09:03:00.125078 21852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:03:00.133684 21852 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:03:00.135286 21852 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:03:00.143136 21852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b720c3203206420084a5aef3f46413e6: Generated new TSK 0
I20260806 09:03:00.143991 21852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:03:00.152060 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21733
W20260806 09:03:00.153201 21822 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36433
W20260806 09:03:00.153970 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21733: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:03:00.154129 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21733
2026-08-06T09:03:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:03:00 dist-test-slave-jpj1 krb5kdc[21870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:03:00 dist-test-slave-jpj1 krb5kdc[21870](info): set up 2 sockets
Aug 06 09:03:00 dist-test-slave-jpj1 krb5kdc[21870](info): commencing operation
krb5kdc: starting...
W20260806 09:03:01.370929 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:03:01 dist-test-slave-jpj1 krb5kdc[21870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:03:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:03:01.727730 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:46161
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:32779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:46161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:03:02.096766 21886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:02.097115 21886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:02.097262 21886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:02.104815 21886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:02.104960 21886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:02.105041 21886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:02.105134 21886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:02.105193 21886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:02.115504 21886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:32779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:46161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:02.116950 21886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:02.118326 21886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:02.130566 21893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:02.130857 21894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:02.130992 21896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:02.131866 21886 server_base.cc:1034] running on GCE node
I20260806 09:03:02.132678 21886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:02.134315 21886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:02.135564 21886 hybrid_clock.cc:648] HybridClock initialized: now 1786006982135476 us; error 65 us; skew 500 ppm
I20260806 09:03:02.135970 21886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:02 dist-test-slave-jpj1 krb5kdc[21870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006982, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:02.139111 21886 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:03:02.140789 21886 webserver.cc:460] Webserver started at http://127.22.98.62:46329/ using document root <none> and password file <none>
I20260806 09:03:02.141498 21886 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:02.141621 21886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:02.141919 21886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:02.145061 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "765e4936970148ef99160fb5342f042b"
format_stamp: "Formatted at 2026-08-06 09:03:02 on dist-test-slave-jpj1"
server_key: "c797292a9e0b7311cdb42f610d939a8c"
server_key_iv: "ba6c32f86ccbc094b0a16f7bb135de11"
server_key_version: "encryptionkey@0"
I20260806 09:03:02.145848 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "765e4936970148ef99160fb5342f042b"
format_stamp: "Formatted at 2026-08-06 09:03:02 on dist-test-slave-jpj1"
server_key: "c797292a9e0b7311cdb42f610d939a8c"
server_key_iv: "ba6c32f86ccbc094b0a16f7bb135de11"
server_key_version: "encryptionkey@0"
I20260806 09:03:02.151175 21886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 09:03:02.154922 21906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:02.156375 21886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:03:02.156551 21886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "765e4936970148ef99160fb5342f042b"
format_stamp: "Formatted at 2026-08-06 09:03:02 on dist-test-slave-jpj1"
server_key: "c797292a9e0b7311cdb42f610d939a8c"
server_key_iv: "ba6c32f86ccbc094b0a16f7bb135de11"
server_key_version: "encryptionkey@0"
I20260806 09:03:02.156751 21886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:02.162586 21886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:02.166895 21886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:02.167199 21886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:02.186393 21886 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:46161
I20260806 09:03:02.186456 21991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:46161 every 8 connection(s)
I20260806 09:03:02.188098 21886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:03:02.192292 21992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:02.192835 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 21886
I20260806 09:03:02.193111 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:03:02.193606 22920 external_mini_cluster.cc:1428] Setting key edbd0300b421593be79e054b27b9b0a6
I20260806 09:03:02.203090 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:02.195653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44123 (local address 127.22.98.62:46161)
0806 09:03:02.196507 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:02.196523 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:02.196577 (+ 54us) server_negotiation.cc:407] Connection header received
0806 09:03:02.199092 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:02.199127 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:02.199655 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:02.200143 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:02.201105 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:02.201113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:02.201130 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:03:02.201147 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:02.201227 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:03:02.201259 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:02.201471 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:02.201605 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 09:03:02.201814 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":320,"threads_started":1}
I20260806 09:03:02.207268 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:03:02.207618 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b: Bootstrap starting.
I20260806 09:03:02.211758 21992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:02.213120 21992 log.cc:824] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:02.215297 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:02.212639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53589 (local address 127.22.98.62:46161)
0806 09:03:02.212831 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:02.212838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:02.213216 (+ 378us) server_negotiation.cc:407] Connection header received
0806 09:03:02.213383 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:02.213387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:02.213440 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:02.213530 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:02.214483 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:02.214488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:02.214491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:03:02.214507 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:02.214548 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:03:02.214564 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:02.214663 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:02.214945 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 09:03:02.215012 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:03:02.216334 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b: No bootstrap required, opened a new log
I20260806 09:03:02.224118 21992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } }
I20260806 09:03:02.224502 21992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:03:02.224604 21992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765e4936970148ef99160fb5342f042b, State: Initialized, Role: FOLLOWER
I20260806 09:03:02.225181 21992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } }
I20260806 09:03:02.225407 21992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:03:02.225546 21992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:03:02.225711 21992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:02.228089 21992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } }
I20260806 09:03:02.228629 21992 leader_election.cc:302] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765e4936970148ef99160fb5342f042b; no voters:
I20260806 09:03:02.229240 21992 leader_election.cc:288] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:03:02.229399 22005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:03:02.230687 22005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [term 1 LEADER]: Becoming Leader. State: Replica: 765e4936970148ef99160fb5342f042b, State: Running, Role: LEADER
I20260806 09:03:02.231256 22005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } }
I20260806 09:03:02.231562 21992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:02.234277 22007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765e4936970148ef99160fb5342f042b. Latest consensus state: current_term: 1 leader_uuid: "765e4936970148ef99160fb5342f042b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } } }
I20260806 09:03:02.234704 22007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:02.234568 22006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "765e4936970148ef99160fb5342f042b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765e4936970148ef99160fb5342f042b" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 46161 } } }
I20260806 09:03:02.234854 22006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:02.236814 22013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:03:02.241849 22013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:03:02.251368 22013 catalog_manager.cc:1347] Generated new cluster ID: d17ad2fd893341bbae2d1e032b56217a
I20260806 09:03:02.251509 22013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:03:02.275061 22013 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:03:02.275519 22013 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:03:02.282912 22013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 765e4936970148ef99160fb5342f042b: Generated new TSK 0
I20260806 09:03:02.283851 22013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:03:02.299952 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21886
W20260806 09:03:02.301002 21987 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44123
W20260806 09:03:02.301683 22920 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 21886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:03:02.301816 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 21886
2026-08-06T09:03:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2148 ms)
[----------] 18 tests from AuthTokenIssuingTest (39090 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:03:02 dist-test-slave-jpj1 krb5kdc[22031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:03:02 dist-test-slave-jpj1 krb5kdc[22031](info): set up 2 sockets
Aug 06 09:03:02 dist-test-slave-jpj1 krb5kdc[22031](info): commencing operation
krb5kdc: starting...
W20260806 09:03:03.521772 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.006s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:03:03 dist-test-slave-jpj1 krb5kdc[22031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:03:03.862640 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37295
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:03:04.219216 22050 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:04.219604 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:04.219705 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:04.219791 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:04.227200 22050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:04.227306 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:04.227365 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:04.227417 22050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:04.227481 22050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:04.237922 22050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:04.239259 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:04.240662 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:04.252468 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:04.252528 22061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:04.252504 22059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:04.253607 22050 server_base.cc:1034] running on GCE node
I20260806 09:03:04.254485 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:04.256100 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:04.257364 22050 hybrid_clock.cc:648] HybridClock initialized: now 1786006984257302 us; error 35 us; skew 500 ppm
I20260806 09:03:04.257733 22050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:04 dist-test-slave-jpj1 krb5kdc[22031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:04.264050 22050 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:03:04.265705 22050 webserver.cc:460] Webserver started at http://127.22.98.62:42051/ using document root <none> and password file <none>
I20260806 09:03:04.266420 22050 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:04.266515 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:04.266803 22050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:04.270015 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "9f3226ad73174e7ca2d8316b8b84a06f"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "9c84d85a4bc4109a62021480d083406c"
server_key_iv: "1a25b30d87d12d4b828bd2c6e65e026a"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.270813 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "9f3226ad73174e7ca2d8316b8b84a06f"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "9c84d85a4bc4109a62021480d083406c"
server_key_iv: "1a25b30d87d12d4b828bd2c6e65e026a"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.276510 22050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:03:04.280405 22069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:04.281944 22050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:03:04.282096 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "9f3226ad73174e7ca2d8316b8b84a06f"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "9c84d85a4bc4109a62021480d083406c"
server_key_iv: "1a25b30d87d12d4b828bd2c6e65e026a"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.282332 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:04.294968 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:04.299065 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:04.299306 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:04.318252 22050 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37295
I20260806 09:03:04.318265 22142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37295 every 8 connection(s)
I20260806 09:03:04.319779 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:03:04.323973 22143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:04.326992 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22050
I20260806 09:03:04.327214 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:03:04.328097 22920 external_mini_cluster.cc:1428] Setting key b6aef27061ee3ab048283eaafaa96a46
I20260806 09:03:04.332754 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:04 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:04.358529 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.336853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37295 (local address 127.0.0.1:60312)
0806 09:03:04.338152 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.338206 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:04.340080 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:04.341042 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:04.341047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:04.341652 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:04.342006 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.342024 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.343960 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.343965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.345632 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.345649 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.346711 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.346723 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.346908 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.347633 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:04.347663 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:04.350311 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:04.354354 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.354370 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.354382 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.354816 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.355303 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.355307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.355309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.355492 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.356455 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:04.356462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:04.356715 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 09:03:04.356957 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 09:03:04.357378 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"spinlock_wait_cycles":18688,"thread_start_us":798,"threads_started":1}
I20260806 09:03:04.358580 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.336838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.22.98.62:37295)
0806 09:03:04.337754 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:04.337777 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:04.338346 (+ 569us) server_negotiation.cc:407] Connection header received
0806 09:03:04.340468 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:04.340476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:04.340787 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:04.341001 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:04.342222 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.343793 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.345788 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.346409 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.346547 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.350600 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:04.350613 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:04.350632 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:04.350670 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:04.354216 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.354953 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.354962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.354965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.355153 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.355630 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.355635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.355638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.356296 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:04.356518 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:04.357193 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 09:03:04.357549 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":665,"thread_start_us":417,"threads_started":1}
I20260806 09:03:04.362016 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:04.363420 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.362900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58306)
0806 09:03:04.363113 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.363256 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 09:03:04.363826 22071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.98.61@KRBTEST.COM; defaulting to no policy
I20260806 09:03:04.365406 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.365022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58318)
0806 09:03:04.365165 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.365237 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:03:04.365595 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34507: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
I20260806 09:03:04.404292 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:34507
--webserver_interface=127.22.98.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:03:04.414940 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } attempt: 1
I20260806 09:03:04.416234 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.415728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58330)
0806 09:03:04.415961 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.416068 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 09:03:04.417354 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.416970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58332)
0806 09:03:04.417113 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.417210 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:03:04.417557 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34507: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
I20260806 09:03:04.485865 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } attempt: 2
I20260806 09:03:04.487270 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.486630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58340)
0806 09:03:04.486904 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.487021 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 09:03:04.488412 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.487971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58354)
0806 09:03:04.488146 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.488253 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 09:03:04.488651 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34507: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
I20260806 09:03:04.579993 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } attempt: 3
I20260806 09:03:04.581418 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.580869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58362)
0806 09:03:04.581108 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.581238 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 09:03:04.582564 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.582149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58376)
0806 09:03:04.582307 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.582394 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:03:04.582839 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34507: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
I20260806 09:03:04.726104 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } attempt: 4
I20260806 09:03:04.727458 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.726784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58390)
0806 09:03:04.727140 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.727265 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 09:03:04.728616 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.728257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58404)
0806 09:03:04.728406 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.728495 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:03:04.728909 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:34507: Network error: Client connection negotiation failed: client connection to 127.22.98.61:34507: connect: Connection refused (error 111)
W20260806 09:03:04.754464 22156 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:04.754835 22156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:04.754935 22156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:04.755049 22156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:04.762459 22156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:04.762581 22156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:04.762668 22156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:04.762748 22156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:04.762844 22156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:04.773092 22156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:34507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.98.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:04.774463 22156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:04.775893 22156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:04.788014 22163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:04.787974 22164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:04.788049 22166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:04.788978 22156 server_base.cc:1034] running on GCE node
I20260806 09:03:04.789839 22156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:04.791371 22156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:04.792665 22156 hybrid_clock.cc:648] HybridClock initialized: now 1786006984792537 us; error 54 us; skew 500 ppm
I20260806 09:03:04.793015 22156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:04 dist-test-slave-jpj1 krb5kdc[22031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:04.796290 22156 init.cc:377] Logged in from keytab as kudu/127.22.98.61@KRBTEST.COM (short username kudu)
I20260806 09:03:04.797729 22156 webserver.cc:460] Webserver started at http://127.22.98.61:37815/ using document root <none> and password file <none>
I20260806 09:03:04.798429 22156 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:04.798529 22156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:04.798859 22156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:04.801992 22156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data/instance:
uuid: "99f0871e68c84db1b32168d35386e391"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "54fd31079a04323a4bff695990719af0"
server_key_iv: "768a0430fcee9e0476c343b05bc3bf37"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.802788 22156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal/instance:
uuid: "99f0871e68c84db1b32168d35386e391"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "54fd31079a04323a4bff695990719af0"
server_key_iv: "768a0430fcee9e0476c343b05bc3bf37"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.808097 22156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 09:03:04.811826 22175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:04.813151 22156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:03:04.813326 22156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "99f0871e68c84db1b32168d35386e391"
format_stamp: "Formatted at 2026-08-06 09:03:04 on dist-test-slave-jpj1"
server_key: "54fd31079a04323a4bff695990719af0"
server_key_iv: "768a0430fcee9e0476c343b05bc3bf37"
server_key_version: "encryptionkey@0"
I20260806 09:03:04.813529 22156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:04.828286 22156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:04.832831 22156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:04.833101 22156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:04.851039 22156 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.61:34507
I20260806 09:03:04.851058 22247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.61:34507 every 8 connection(s)
I20260806 09:03:04.852463 22156 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
I20260806 09:03:04.856338 22248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:04.857501 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22156
I20260806 09:03:04.857685 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-1/wal/instance
I20260806 09:03:04.858155 22920 external_mini_cluster.cc:1428] Setting key 7ed71b2db02e181061d54373ba5bb0da
I20260806 09:03:04.863101 22248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:04 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:04.885102 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.865723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.22.98.62:37295)
0806 09:03:04.866247 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:04.866254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:04.867220 (+ 966us) server_negotiation.cc:407] Connection header received
0806 09:03:04.869155 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:04.869161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:04.869241 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:04.869360 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:04.870875 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.871751 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.873863 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.874291 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.874403 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.878724 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:04.878730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:04.878733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:04.878781 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:04.881487 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.882425 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.882431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.882435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.882572 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.883187 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.883192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.883194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.883497 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:04.883677 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:04.884650 (+ 973us) server_negotiation.cc:299] Negotiation successful
0806 09:03:04.884835 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":239,"threads_started":1}
I20260806 09:03:04.886587 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.866239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37295 (local address 127.0.0.1:60326)
0806 09:03:04.867051 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.867098 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:04.868879 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:04.869554 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:04.869561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:04.870097 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:04.870660 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.870678 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.871930 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.871935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.873664 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.873680 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.874524 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.874530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.874882 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.875620 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:04.875646 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:04.878459 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:04.881646 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.881655 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.881675 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.882280 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.882752 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.882757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.882760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.883027 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.883676 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:04.883684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:04.884226 (+ 542us) client_negotiation.cc:769] Sending connection context
0806 09:03:04.884535 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 09:03:04.885075 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":351,"threads_started":1}
I20260806 09:03:04.888504 22248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.98.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.60@KRBTEST.COM" created.
Aug 06 09:03:04 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:04.902979 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.889404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58408)
0806 09:03:04.889650 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.889671 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:04.889828 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:04.890585 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:04.890591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:04.891073 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:04.891270 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.891283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.892778 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.892782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.893691 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:04.893706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.894553 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.894559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:04.894763 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.895330 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:04.895345 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:04.897520 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:04.900541 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.900546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.900550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.900920 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.901285 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:04.901289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:04.901293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:04.901410 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:04.901986 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:04.901990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:04.902149 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 09:03:04.902269 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 09:03:04.902549 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 09:03:04.903057 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.889463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.22.98.61:34507)
0806 09:03:04.890146 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:04.890153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:04.890181 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:03:04.890228 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:04.890235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:04.890413 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:04.890530 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:04.891471 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.892624 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.893835 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:04.894416 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:04.894530 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:04.897730 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:04.897747 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:04.897763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:04.897792 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:04.900416 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.901042 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.901049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.901051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.901167 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:04.901505 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:04.901510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:04.901514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:04.901865 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:04.902065 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:04.902448 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 09:03:04.902675 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":374,"threads_started":1}
I20260806 09:03:04.905943 22248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:04.907178 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.906726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:42970)
0806 09:03:04.906901 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.907015 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 09:03:04.907485 22178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:04.910460 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.910013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:42974)
0806 09:03:04.910203 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.910312 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 09:03:04.910684 22248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
I20260806 09:03:04.932289 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:41267
--webserver_interface=127.22.98.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:03:04.960043 22248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 1
I20260806 09:03:04.961468 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.960884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:42988)
0806 09:03:04.961122 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.961270 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 09:03:04.962586 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:04.962161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:42998)
0806 09:03:04.962331 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:04.962449 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 09:03:04.962798 22248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.028187 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } attempt: 5
I20260806 09:03:05.031033 22248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 2
I20260806 09:03:05.032258 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.031659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43008)
0806 09:03:05.031945 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.032049 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 09:03:05.033346 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.032989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43016)
0806 09:03:05.033137 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.033207 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:03:05.033631 22248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:05.042825 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.028973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58410)
0806 09:03:05.029213 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.029235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.029377 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.029750 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.029754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.030304 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.030554 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.030569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.031757 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.031763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.032803 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.032822 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.033868 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.033873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.034040 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.034616 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.034635 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.036649 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.040561 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.040565 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.040569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.040945 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.041356 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.041360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.041362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.041497 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.042072 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.042076 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.042225 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.042337 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.042577 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 09:03:05.042985 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.029142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.22.98.61:34507)
0806 09:03:05.029300 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.029304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.029324 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:03:05.029502 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.029506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.029601 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.029684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.030726 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.031586 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.032951 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.033732 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.033860 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.036857 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.036864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.036869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.036908 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.040415 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.041071 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.041077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.041080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.041246 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.041595 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.041601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.041604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.041972 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.042122 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.042464 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.042689 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 09:03:05.044018 22143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:05.045122 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.044745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43032)
0806 09:03:05.044915 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.044988 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:03:05.046098 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.045764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43048)
0806 09:03:05.045920 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.045986 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 09:03:05.046303 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.097536 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 1
I20260806 09:03:05.098881 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.098355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43060)
0806 09:03:05.098573 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.098671 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 09:03:05.099947 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.099609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43074)
0806 09:03:05.099757 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.099833 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 09:03:05.100212 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.124850 22248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 3
I20260806 09:03:05.125985 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.125487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43078)
0806 09:03:05.125696 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.125822 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 09:03:05.127251 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.126748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43082)
0806 09:03:05.127033 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.127121 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260806 09:03:05.127487 22248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.168455 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 2
I20260806 09:03:05.169948 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.169325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43098)
0806 09:03:05.169605 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.169765 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 09:03:05.171125 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.170689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43106)
0806 09:03:05.170887 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.170976 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 09:03:05.171375 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.270753 22248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 4
I20260806 09:03:05.271958 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.271451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43110)
0806 09:03:05.271700 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.271798 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 09:03:05.273046 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.272618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43120)
0806 09:03:05.272816 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.272907 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 09:03:05.273346 22248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
I20260806 09:03:05.277590 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 3
I20260806 09:03:05.278551 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.278135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43130)
0806 09:03:05.278311 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.278396 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 09:03:05.279510 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.279122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43142)
0806 09:03:05.279325 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.279400 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 09:03:05.279896 22143 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:41267: Network error: Client connection negotiation failed: client connection to 127.22.98.60:41267: connect: Connection refused (error 111)
W20260806 09:03:05.289482 22257 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:05.289853 22257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:05.289955 22257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:05.290046 22257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:05.297465 22257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:05.297564 22257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:05.297699 22257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:05.297770 22257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:05.297855 22257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:05.307964 22257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37295,127.22.98.61:34507,127.22.98.60:41267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:41267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.98.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:05.309310 22257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:05.310797 22257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:05.324851 22262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:05.324699 22264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:05.325434 22266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:05.326074 22257 server_base.cc:1034] running on GCE node
I20260806 09:03:05.327082 22257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:05.328991 22257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:05.330292 22257 hybrid_clock.cc:648] HybridClock initialized: now 1786006985330215 us; error 44 us; skew 500 ppm
I20260806 09:03:05.330695 22257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006985, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:05.334296 22257 init.cc:377] Logged in from keytab as kudu/127.22.98.60@KRBTEST.COM (short username kudu)
I20260806 09:03:05.335958 22257 webserver.cc:460] Webserver started at http://127.22.98.60:42291/ using document root <none> and password file <none>
I20260806 09:03:05.336660 22257 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:05.336761 22257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:05.337052 22257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:05.340286 22257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data/instance:
uuid: "19dc62260aba4ce2940e406575642ce3"
format_stamp: "Formatted at 2026-08-06 09:03:05 on dist-test-slave-jpj1"
server_key: "5647cf646355b322520b15e6f8e9b1b1"
server_key_iv: "92c8202257b96265a18d52372e2dc475"
server_key_version: "encryptionkey@0"
I20260806 09:03:05.341104 22257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal/instance:
uuid: "19dc62260aba4ce2940e406575642ce3"
format_stamp: "Formatted at 2026-08-06 09:03:05 on dist-test-slave-jpj1"
server_key: "5647cf646355b322520b15e6f8e9b1b1"
server_key_iv: "92c8202257b96265a18d52372e2dc475"
server_key_version: "encryptionkey@0"
I20260806 09:03:05.346697 22257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:03:05.350554 22273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:05.352200 22257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:03:05.352365 22257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "19dc62260aba4ce2940e406575642ce3"
format_stamp: "Formatted at 2026-08-06 09:03:05 on dist-test-slave-jpj1"
server_key: "5647cf646355b322520b15e6f8e9b1b1"
server_key_iv: "92c8202257b96265a18d52372e2dc475"
server_key_version: "encryptionkey@0"
I20260806 09:03:05.352567 22257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:05.365940 22257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:05.370146 22257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:05.370383 22257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:05.388553 22257 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.60:41267
I20260806 09:03:05.388572 22341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.60:41267 every 8 connection(s)
I20260806 09:03:05.389961 22257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
I20260806 09:03:05.393824 22343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:05.395157 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22257
I20260806 09:03:05.395339 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786006610327111-22920-0/minicluster-data/master-2/wal/instance
I20260806 09:03:05.395891 22920 external_mini_cluster.cc:1428] Setting key 7c6de54e497f990878213fccd2c39b9b
I20260806 09:03:05.403285 22343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006985, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:05.421188 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.397811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.22.98.60:41267)
0806 09:03:05.398622 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.398634 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.398680 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:03:05.400292 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.400317 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.400710 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.401047 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.402915 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.405016 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.406766 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.407904 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.408134 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.413331 (+ 5197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.413340 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.413352 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.413395 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.416806 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.417625 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.417632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.417640 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.417859 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.418307 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.418313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.418316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.418902 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.419214 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.419681 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.420131 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":373,"threads_started":1}
I20260806 09:03:05.426646 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.406686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.22.98.62:37295)
0806 09:03:05.411954 (+ 5268us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.411959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.411977 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:03:05.412032 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.412035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.412121 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.412233 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.413862 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.414924 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.416583 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.417182 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.417286 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.420621 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.420627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.420630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.420664 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.423642 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.424427 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.424433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.424437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.424612 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.425073 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.425077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.425080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.425471 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.425627 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.426164 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.426365 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5161,"thread_start_us":5016,"threads_started":1}
I20260806 09:03:05.426838 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.406462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37295 (local address 127.0.0.1:60338)
0806 09:03:05.407704 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.407771 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.407925 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.412361 (+ 4436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.412370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.413309 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.413630 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.413640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.415117 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.415123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.416406 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.416421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.417375 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.417382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.417569 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.418210 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.418241 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.420431 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.423777 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.423789 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.423796 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.424295 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.424725 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.424730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.424733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.424927 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.425595 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.425603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.425825 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.426052 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.426422 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":763,"threads_started":1}
I20260806 09:03:05.428578 22343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006985, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:05.441264 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.424563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.22.98.62:37295)
0806 09:03:05.425229 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.425234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.425256 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:03:05.425302 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.425306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.425371 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.425479 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.427120 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.428152 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.429691 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.430122 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.430235 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.435184 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.435211 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.435215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.435250 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.438590 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.439249 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.439255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.439259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.439421 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.439800 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.439805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.439809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.440111 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.440314 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.440699 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.440876 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":279,"threads_started":1}
I20260806 09:03:05.442476 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.429390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:34507 (local address 127.0.0.1:58416)
0806 09:03:05.429552 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.429572 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.429684 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.430012 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.430017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.430375 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.430526 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.430534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.431614 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.431620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.432589 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.432601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.433342 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.433347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.433480 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.434039 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.434061 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.436590 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.439881 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.439885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.439887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.440284 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.440668 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.440673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.440676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.440861 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.441558 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.441562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.441688 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.441820 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.442104 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 09:03:05.442507 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.429508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.22.98.61:34507)
0806 09:03:05.429712 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.429718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.429771 (+ 53us) server_negotiation.cc:407] Connection header received
0806 09:03:05.429831 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.429835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.429895 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.429976 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.430676 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.431455 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.432722 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.433184 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.433314 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.436804 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.436811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.436816 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.436859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.439749 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.440404 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.440409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.440411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.440540 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.440980 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.440986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.440990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.441343 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.441500 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.441941 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.442141 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:03:05.443671 22343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006985, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:05.457154 22143 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 4
I20260806 09:03:05.458334 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.444461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43154)
0806 09:03:05.444756 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.444773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.444881 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.445191 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.445196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.445554 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.445686 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.445739 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.446800 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.446806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.448027 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.448039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.448863 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.448868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.449426 (+ 558us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.450047 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.450069 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.452504 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.455510 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.455516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.455520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.456038 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.456451 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.456456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.456460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.456637 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.457232 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.457237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.457408 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.457568 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.457902 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":268}
I20260806 09:03:05.458969 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.444461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.22.98.60:41267)
0806 09:03:05.444643 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.444649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.444878 (+ 229us) server_negotiation.cc:407] Connection header received
0806 09:03:05.444988 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.444991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.445048 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.445155 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.445868 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.446635 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.448161 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.448681 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.449124 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.452694 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.452700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.452704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.452752 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.455372 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.456182 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.456186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.456189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.456312 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.456737 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.456742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.456746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.457098 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.457248 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.457703 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.458442 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":523,"server-negotiator.queue_time_us":61}
I20260806 09:03:05.459653 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.443171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.22.98.61:34507)
0806 09:03:05.443331 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.443336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.443353 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:03:05.443510 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.443513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.443605 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.443756 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.445180 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.446085 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.447714 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.448169 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.448327 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.453517 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.453524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.453529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.453572 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.456511 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.457349 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.457354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.457358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.457513 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.457980 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.457985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.457989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.458337 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.458621 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.459084 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.459316 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:03:05.464946 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:03:05.468497 22343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: Bootstrap starting.
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:03:05.468281 22110 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9f3226ad73174e7ca2d8316b8b84a06f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60344
I20260806 09:03:05.472714 22343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:05.473870 22343 log.cc:824] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:05.476950 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.457814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43168)
0806 09:03:05.458001 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.458021 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.458143 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.460775 (+ 2632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.460780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.461298 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.461521 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.461537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.463151 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.463156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.464595 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.464612 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.465667 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.465673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.465872 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.466642 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.466670 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.469871 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.474017 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.474023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.474026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.474576 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.474955 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.474960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.474964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.475116 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.475932 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.475938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.476104 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.476261 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.476570 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 09:03:05.477000 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.458275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.22.98.60:41267)
0806 09:03:05.460425 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.460431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.460464 (+ 33us) server_negotiation.cc:407] Connection header received
0806 09:03:05.460516 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.460521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.460591 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.461676 (+ 1085us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.461801 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.462964 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.464813 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.465491 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.466200 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.470096 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.470104 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.470109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.470151 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.473854 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.474694 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.474699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.474703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.474840 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.475236 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.475239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.475242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.475704 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.475866 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.476377 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.476581 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2021,"thread_start_us":9428,"threads_started":1}
I20260806 09:03:05.477360 22343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: No bootstrap required, opened a new log
I20260806 09:03:05.483014 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: Bootstrap starting.
I20260806 09:03:05.487268 22143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:05.487812 22343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.488253 22343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19dc62260aba4ce2940e406575642ce3, State: Initialized, Role: FOLLOWER
I20260806 09:03:05.488706 22143 log.cc:824] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:05.488796 22343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.489799 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.490206 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.491122 22343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:05.492630 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: No bootstrap required, opened a new log
I20260806 09:03:05.500854 22143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.501364 22143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3226ad73174e7ca2d8316b8b84a06f, State: Initialized, Role: FOLLOWER
I20260806 09:03:05.501976 22143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.503111 22373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.503609 22373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.505035 22143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:05.505235 22110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 09:03:05.505407 22110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 0 FOLLOWER]: Advancing to term 1
W20260806 09:03:05.507742 22371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:03:05.507871 22371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:03:05.509016 22110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.511698 22110 leader_election.cc:288] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [CANDIDATE]: Term 1 election: Requested vote from peers 99f0871e68c84db1b32168d35386e391 (127.22.98.61:34507), 19dc62260aba4ce2940e406575642ce3 (127.22.98.60:41267)
I20260806 09:03:05.511636 22305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "19dc62260aba4ce2940e406575642ce3"
I20260806 09:03:05.512132 22305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:05.515940 22305 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3226ad73174e7ca2d8316b8b84a06f in term 1.
I20260806 09:03:05.516754 22072 leader_election.cc:302] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19dc62260aba4ce2940e406575642ce3, 9f3226ad73174e7ca2d8316b8b84a06f; no voters:
I20260806 09:03:05.518409 22373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:03:05.511500 22257 master_runner.cc:186] Error getting master registration for 127.22.98.62:37295: OK, instance_id { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" instance_seqno: 1786006984312993 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:03:05.522805 22257 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:03:05.536232 22373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [term 1 LEADER]: Becoming Leader. State: Replica: 9f3226ad73174e7ca2d8316b8b84a06f, State: Running, Role: LEADER
I20260806 09:03:05.537071 22373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.541213 22379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3226ad73174e7ca2d8316b8b84a06f. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.541697 22379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: This master's current role is: LEADER
W20260806 09:03:05.552290 22078 tablet.cc:2369] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:03:05.555254 22391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:03:05.567566 22391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:03:05.572558 22248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } attempt: 5
Aug 06 09:03:05 dist-test-slave-jpj1 krb5kdc[22031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006984, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:03:05.591449 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.573351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:41267 (local address 127.0.0.1:43172)
0806 09:03:05.573882 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:05.573905 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:05.574043 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:05.575031 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:05.575036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:05.575606 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:05.575838 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.575852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.577292 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.577298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.578764 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:05.578780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.580092 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.580099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:05.580325 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.581032 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:05.581056 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:05.584101 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:05.588457 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.588462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.588466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.589024 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.589495 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:05.589500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:05.589504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:05.589709 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:05.590407 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:05.590412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:05.590572 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 09:03:05.590715 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 09:03:05.591037 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377}
I20260806 09:03:05.592677 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.574304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.22.98.60:41267)
0806 09:03:05.574475 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.574481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.574500 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:03:05.574563 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.574568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.574687 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.574817 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.576027 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.577111 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.578953 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.579729 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.579861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.584343 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.584351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.584355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.584396 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.588247 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.589179 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.589185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.589188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.589357 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.589853 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.589858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.589862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.590206 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.590345 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.592013 (+ 1668us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.592259 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 09:03:05.594321 22401 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 09:03:05.603050 22305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [term 1 FOLLOWER]: Refusing update from remote peer 9f3226ad73174e7ca2d8316b8b84a06f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:03:05.613713 22373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [LEADER]: Connected to new peer: Peer: permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:03:05.619733 22248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: Bootstrap starting.
W20260806 09:03:05.622494 22282 tablet.cc:2369] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:03:05.606465 22050 master_runner.cc:186] Error getting master registration for 127.22.98.61:34507: OK, instance_id { permanent_uuid: "99f0871e68c84db1b32168d35386e391" instance_seqno: 1786006984845644 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:03:05.665757 22050 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:03:05.667294 22248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:05.669265 22248 log.cc:824] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:05.679068 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3226ad73174e7ca2d8316b8b84a06f. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.679466 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.694959 22379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.695298 22379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:05.696683 22355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.696990 22355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.698378 22391 catalog_manager.cc:1347] Generated new cluster ID: c6040df653bc4dd1a4449509fec89aa8
I20260806 09:03:05.698491 22391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:03:05.705443 22248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: No bootstrap required, opened a new log
I20260806 09:03:05.721323 22351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.598330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.22.98.62:37295)
0806 09:03:05.598631 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.598637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.689921 (+ 91284us) server_negotiation.cc:407] Connection header received
0806 09:03:05.690230 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.690237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.690339 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.690452 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.706395 (+ 15943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.707527 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.709601 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.710368 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.710495 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.714026 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.714033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.714039 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.714084 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.717777 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.718659 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.718665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.718669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.718851 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.719380 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.719386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.719390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.719818 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.720087 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.720719 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.720960 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":168}
I20260806 09:03:05.727429 22248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.729810 22248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0871e68c84db1b32168d35386e391, State: Initialized, Role: FOLLOWER
I20260806 09:03:05.730446 22248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } }
I20260806 09:03:05.733737 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.599297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.22.98.61:34507)
0806 09:03:05.600607 (+ 1310us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.600614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.674255 (+ 73641us) server_negotiation.cc:407] Connection header received
0806 09:03:05.674611 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.674618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.674713 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.674793 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.699956 (+ 25163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.701072 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.707656 (+ 6584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.708310 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.708414 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.724249 (+ 15835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.724258 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.724263 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.724308 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.728138 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.729633 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.729639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.729643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.729894 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.730834 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.730840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.730844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.731226 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.731516 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.733079 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.733369 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169}
I20260806 09:03:05.746474 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:05.600390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.22.98.60:41267)
0806 09:03:05.611820 (+ 11430us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.611828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:05.711396 (+ 99568us) server_negotiation.cc:407] Connection header received
0806 09:03:05.712835 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:05.712845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:05.712930 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:05.713003 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:05.717198 (+ 4195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.718316 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.727689 (+ 9373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:05.728382 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:05.728506 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:05.735891 (+ 7385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:05.735898 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:05.735904 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:05.735967 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:05.742760 (+ 6793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.743757 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.743764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.743768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.743929 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:05.744425 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:05.744431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:05.744435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:05.744834 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:05.745099 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:05.745748 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 09:03:05.746024 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11211}
I20260806 09:03:05.747466 22248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:05.747938 22211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:05.751688 22213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99f0871e68c84db1b32168d35386e391"
I20260806 09:03:05.752889 22211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [term 1 FOLLOWER]: Refusing update from remote peer 9f3226ad73174e7ca2d8316b8b84a06f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:03:05.756974 22213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3226ad73174e7ca2d8316b8b84a06f in term 1.
I20260806 09:03:05.759891 22373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:03:05.747027 22412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.763686 22412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.776897 22391 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:03:05.778928 22391 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:03:05.803810 22391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f: Generated new TSK 0
I20260806 09:03:05.805033 22391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:03:05.809262 22421 mvcc.cc:205] Tried to move back new op lower bound from 7315484613001179136 to 7315484612766240768. Current Snapshot: MvccSnapshot[applied={T|T < 7315484613001179136}]
I20260806 09:03:05.816185 22412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f3226ad73174e7ca2d8316b8b84a06f. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.816525 22412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.821045 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.821506 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:05.836895 22373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3226ad73174e7ca2d8316b8b84a06f" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37295 } } peers { permanent_uuid: "99f0871e68c84db1b32168d35386e391" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 34507 } } peers { permanent_uuid: "19dc62260aba4ce2940e406575642ce3" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 41267 } } }
I20260806 09:03:05.837308 22373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3226ad73174e7ca2d8316b8b84a06f [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:05.894819 22429 catalog_manager.cc:1259] Loaded cluster ID: c6040df653bc4dd1a4449509fec89aa8
I20260806 09:03:05.903684 22429 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: loading cluster ID for follower catalog manager: success
I20260806 09:03:05.908141 22429 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: acquiring CA information for follower catalog manager: success
I20260806 09:03:05.916695 22429 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99f0871e68c84db1b32168d35386e391: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 09:03:05.962877 22920 mini_kdc.cc:348] Time spent kdestroy: real 0.136s user 0.006s sys 0.024s
W20260806 09:03:06.023083 22254 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:05.996788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.22.98.61:34507)
0806 09:03:05.997001 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:05.997008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.018788 (+ 21780us) server_negotiation.cc:407] Connection header received
0806 09:03:06.021330 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.021337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.021457 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.021536 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.022818 (+ 1282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58448: BlockingRecv error: recv got EOF from 127.0.0.1:58448 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 09:03:06.025480 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.003574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.22.98.60:41267)
0806 09:03:06.003782 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.003788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.020970 (+ 17182us) server_negotiation.cc:407] Connection header received
0806 09:03:06.023880 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.023885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.023984 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.024058 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.025187 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43196: BlockingRecv error: recv got EOF from 127.0.0.1:43196 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 09:03:06.028942 22351 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.014991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.22.98.62:37295)
0806 09:03:06.015254 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.015261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.026334 (+ 11073us) server_negotiation.cc:407] Connection header received
0806 09:03:06.027482 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.027488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.027598 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.027690 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.028682 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60358: BlockingRecv error: recv got EOF from 127.0.0.1:60358 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 09:03:06.080257 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.060495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.22.98.61:34507)
0806 09:03:06.060709 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.060715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.071425 (+ 10710us) server_negotiation.cc:407] Connection header received
0806 09:03:06.072121 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.072129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.072383 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.072559 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.073363 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.074535 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.076743 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.077436 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.077556 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.078452 (+ 896us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.079302 (+ 850us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.079442 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.079757 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.079881 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 09:03:06.122239 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.111859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.22.98.60:41267)
0806 09:03:06.112166 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.112173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.119878 (+ 7705us) server_negotiation.cc:407] Connection header received
0806 09:03:06.120513 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.120521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.120681 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.120770 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.121965 (+ 1195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43198: BlockingRecv error: recv got EOF from 127.0.0.1:43198 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 09:03:06.173312 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.156836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.22.98.61:34507)
0806 09:03:06.157035 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.157042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.166248 (+ 9206us) server_negotiation.cc:407] Connection header received
0806 09:03:06.166789 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.166796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.166930 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.166998 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.167727 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.168833 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.170895 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.171643 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.171760 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.172401 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.172593 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.172657 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.172902 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.173001 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 09:03:06.197971 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.191508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.22.98.60:41267)
0806 09:03:06.191737 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.191744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.196261 (+ 4517us) server_negotiation.cc:407] Connection header received
0806 09:03:06.196505 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.196511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.196620 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.196750 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.197733 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 09:03:06.211050 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.204003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.22.98.61:34507)
0806 09:03:06.204200 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.204207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.204760 (+ 553us) server_negotiation.cc:407] Connection header received
0806 09:03:06.205025 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.205031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.205127 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.205225 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.205671 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.206872 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.208828 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.209475 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.209657 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.210142 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.210290 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.210398 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.210596 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.210683 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 09:03:06.222527 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.220787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.22.98.60:41267)
0806 09:03:06.221014 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.221021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.221088 (+ 67us) server_negotiation.cc:407] Connection header received
0806 09:03:06.221304 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.221310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.221406 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.221523 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.222272 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43214: BlockingRecv error: recv got EOF from 127.0.0.1:43214 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:03:06.236117 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.229798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.22.98.61:34507)
0806 09:03:06.230062 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.230072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.230310 (+ 238us) server_negotiation.cc:407] Connection header received
0806 09:03:06.230484 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.230490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.230577 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.230711 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.231225 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.232479 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.233922 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.234637 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.234781 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.235229 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.235390 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.235494 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.235736 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.235828 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
W20260806 09:03:06.244506 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.242457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.22.98.60:41267)
0806 09:03:06.242777 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.242784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.242956 (+ 172us) server_negotiation.cc:407] Connection header received
0806 09:03:06.243130 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.243137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.243247 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.243375 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.244255 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43230: BlockingRecv error: recv got EOF from 127.0.0.1:43230 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
I20260806 09:03:06.261952 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.255917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58512 (local address 127.22.98.61:34507)
0806 09:03:06.256131 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.256137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.256644 (+ 507us) server_negotiation.cc:407] Connection header received
0806 09:03:06.256849 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.256854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.256931 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.257017 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.257547 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.258383 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.260054 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.260566 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.260700 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.261144 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.261273 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.261360 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.261525 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.261614 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 09:03:06.274529 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.272411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.22.98.60:41267)
0806 09:03:06.272633 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.272640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.273004 (+ 364us) server_negotiation.cc:407] Connection header received
0806 09:03:06.273185 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.273189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.273267 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.273358 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.274250 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43246: BlockingRecv error: recv got EOF from 127.0.0.1:43246 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 09:03:06.301095 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.294809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.22.98.61:34507)
0806 09:03:06.295043 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.295049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.295444 (+ 395us) server_negotiation.cc:407] Connection header received
0806 09:03:06.295670 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.295676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.295776 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.295877 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.296292 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.297525 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.298914 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.299660 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.299807 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.300269 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.300430 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.300534 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.300716 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.300808 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 09:03:06.309510 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.307475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.22.98.60:41267)
0806 09:03:06.307702 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.307709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.308066 (+ 357us) server_negotiation.cc:407] Connection header received
0806 09:03:06.308221 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.308228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.308364 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.308498 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.309260 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43254: BlockingRecv error: recv got EOF from 127.0.0.1:43254 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 09:03:06.354241 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.348069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.22.98.61:34507)
0806 09:03:06.348320 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.348327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.348731 (+ 404us) server_negotiation.cc:407] Connection header received
0806 09:03:06.348916 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.348920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.349016 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.349102 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.349566 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.350361 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.352316 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.352814 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.352925 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.353457 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.353580 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.353698 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.353834 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.353904 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 09:03:06.362562 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.360427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.22.98.60:41267)
0806 09:03:06.360632 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.360639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.361160 (+ 521us) server_negotiation.cc:407] Connection header received
0806 09:03:06.361315 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.361319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.361395 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.361481 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.362332 (+ 851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43262: BlockingRecv error: recv got EOF from 127.0.0.1:43262 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:03:06.435163 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.430324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.22.98.61:34507)
0806 09:03:06.430518 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.430522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.430832 (+ 310us) server_negotiation.cc:407] Connection header received
0806 09:03:06.431001 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.431005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.431079 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.431169 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.431565 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.432314 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.433600 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.434035 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.434154 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.434534 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.434660 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.434765 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.434877 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.434943 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 09:03:06.442787 22346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:06.441127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.22.98.60:41267)
0806 09:03:06.441396 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.441401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.441538 (+ 137us) server_negotiation.cc:407] Connection header received
0806 09:03:06.441742 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.441746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.441830 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.441913 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:06.442558 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43266: BlockingRecv error: recv got EOF from 127.0.0.1:43266 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 09:03:06.509960 22371 catalog_manager.cc:1259] Loaded cluster ID: c6040df653bc4dd1a4449509fec89aa8
I20260806 09:03:06.510078 22371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: loading cluster ID for follower catalog manager: success
I20260806 09:03:06.512648 22371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: acquiring CA information for follower catalog manager: success
I20260806 09:03:06.514464 22371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19dc62260aba4ce2940e406575642ce3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:03:06.581935 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.574862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.22.98.61:34507)
0806 09:03:06.575054 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.575061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.575499 (+ 438us) server_negotiation.cc:407] Connection header received
0806 09:03:06.575760 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.575767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.575864 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.575964 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.576486 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.577526 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.579913 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.580408 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.580496 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.581033 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.581193 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.581285 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.581503 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.581605 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 09:03:06.593971 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.588130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.22.98.60:41267)
0806 09:03:06.588335 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.588341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.588516 (+ 175us) server_negotiation.cc:407] Connection header received
0806 09:03:06.588667 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.588671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.588829 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.588928 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.589345 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.590073 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.591590 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.592031 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.592203 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.592775 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.593375 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.593498 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.593608 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.593682 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:03:06.611838 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.600579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.22.98.61:34507)
0806 09:03:06.600786 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.600792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.605269 (+ 4477us) server_negotiation.cc:407] Connection header received
0806 09:03:06.605549 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.605554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.605643 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.605708 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.606575 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.607650 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.609820 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.610339 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.610467 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.611043 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.611174 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.611247 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.611448 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.611556 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:03:06.612162 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.600900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.22.98.60:41267)
0806 09:03:06.601095 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.601101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.605936 (+ 4835us) server_negotiation.cc:407] Connection header received
0806 09:03:06.606170 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.606176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.606273 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.606364 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.607054 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.608035 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.610075 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.610723 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.610847 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.611357 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.611482 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.611586 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.611780 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.611866 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:03:06.612839 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:06.600484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.22.98.62:37295)
0806 09:03:06.605378 (+ 4894us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:06.605385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:06.605464 (+ 79us) server_negotiation.cc:407] Connection header received
0806 09:03:06.605710 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:06.605715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:06.605989 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:06.606186 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:06.606956 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.607804 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.610088 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:06.610514 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:06.610645 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:06.611235 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:06.611729 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:06.611831 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:06.612213 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 09:03:06.612540 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4768,"thread_start_us":4653,"threads_started":1}
I20260806 09:03:06.618453 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22050
I20260806 09:03:06.619702 22138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:06.730175 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22050
W20260806 09:03:06.758092 22180 connection.cc:535] server connection from 127.0.0.1:58410 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 09:03:06.758455 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22156
I20260806 09:03:06.759660 22243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:06.867735 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22156
I20260806 09:03:06.894550 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22257
I20260806 09:03:06.895761 22336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:07.001459 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22257
2026-08-06T09:03:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4698 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:03:07 dist-test-slave-jpj1 krb5kdc[22593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:03:07 dist-test-slave-jpj1 krb5kdc[22593](info): set up 2 sockets
Aug 06 09:03:07 dist-test-slave-jpj1 krb5kdc[22593](info): commencing operation
krb5kdc: starting...
W20260806 09:03:08.219887 22920 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/test-user.keytab.
Aug 06 09:03:08 dist-test-slave-jpj1 krb5kdc[22593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.98.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
Entry for principal kudu/127.22.98.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab.
I20260806 09:03:08.557060 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37927
--webserver_interface=127.22.98.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:03:08.909729 22613 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:08.910070 22613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:08.910172 22613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:08.910285 22613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:08.917551 22613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:08.917646 22613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:08.917722 22613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:08.917795 22613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:08.917886 22613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:08.927773 22613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.62.keytab
--principal=kudu/127.22.98.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.62:37927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.98.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:08.929131 22613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:08.930480 22613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:08.942466 22619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:08.942562 22621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:08.942533 22623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:08.943428 22613 server_base.cc:1034] running on GCE node
I20260806 09:03:08.944223 22613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:08.945672 22613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:08.946906 22613 hybrid_clock.cc:648] HybridClock initialized: now 1786006988946835 us; error 39 us; skew 500 ppm
I20260806 09:03:08.947232 22613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:08 dist-test-slave-jpj1 krb5kdc[22593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:08.950369 22613 init.cc:377] Logged in from keytab as kudu/127.22.98.62@KRBTEST.COM (short username kudu)
I20260806 09:03:08.951884 22613 webserver.cc:460] Webserver started at http://127.22.98.62:34479/ using document root <none> and password file <none>
I20260806 09:03:08.952569 22613 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:08.952674 22613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:08.952994 22613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:08.956070 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data/instance:
uuid: "4eeae5d834d447d68f4c5c5be9946be4"
format_stamp: "Formatted at 2026-08-06 09:03:08 on dist-test-slave-jpj1"
server_key: "b8addc46d65a990d5238dcedf38184c4"
server_key_iv: "9902f3251a28199faf80a980f7e50375"
server_key_version: "encryptionkey@0"
I20260806 09:03:08.956877 22613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance:
uuid: "4eeae5d834d447d68f4c5c5be9946be4"
format_stamp: "Formatted at 2026-08-06 09:03:08 on dist-test-slave-jpj1"
server_key: "b8addc46d65a990d5238dcedf38184c4"
server_key_iv: "9902f3251a28199faf80a980f7e50375"
server_key_version: "encryptionkey@0"
I20260806 09:03:08.962083 22613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 09:03:08.965663 22631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:08.967046 22613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:03:08.967221 22613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal
uuid: "4eeae5d834d447d68f4c5c5be9946be4"
format_stamp: "Formatted at 2026-08-06 09:03:08 on dist-test-slave-jpj1"
server_key: "b8addc46d65a990d5238dcedf38184c4"
server_key_iv: "9902f3251a28199faf80a980f7e50375"
server_key_version: "encryptionkey@0"
I20260806 09:03:08.967435 22613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:08.980458 22613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:08.984843 22613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:08.985119 22613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:09.003520 22613 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.62:37927
I20260806 09:03:09.003557 22706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.62:37927 every 8 connection(s)
I20260806 09:03:09.005030 22613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/data/info.pb
I20260806 09:03:09.008991 22709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:09.009965 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22613
I20260806 09:03:09.010176 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-0/wal/instance
I20260806 09:03:09.010789 22920 external_mini_cluster.cc:1428] Setting key 9287f66cfc70b3277812f6c7d9abaeee
I20260806 09:03:09.018095 22709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:09 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:09.042160 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.021963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37927 (local address 127.0.0.1:50696)
0806 09:03:09.022874 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.022922 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:09.024950 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:09.025807 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:09.025813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:09.026525 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:09.026879 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.026898 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.028500 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.028505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.030095 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.030117 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.031113 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.031120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.031278 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.031992 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:09.032031 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:09.034737 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:09.037930 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.037944 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.037956 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.038506 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.038925 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.038929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.038932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.039154 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.040128 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:09.040135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:09.040362 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 09:03:09.040586 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 09:03:09.041024 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":384,"threads_started":1}
I20260806 09:03:09.042184 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.021975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.22.98.62:37927)
0806 09:03:09.023675 (+ 1700us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:09.023683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:09.023744 (+ 61us) server_negotiation.cc:407] Connection header received
0806 09:03:09.025337 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:09.025343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:09.025580 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:09.025770 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:09.027084 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.028355 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.030264 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.030852 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.030995 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.034981 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:09.034991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:09.035004 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:09.035038 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:09.037795 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.038623 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.038630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.038633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.038814 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.039277 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.039282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.039284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.039977 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:09.040220 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:09.040737 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 09:03:09.041163 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":1473,"thread_start_us":1251,"threads_started":1}
I20260806 09:03:09.045547 22709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:09.046526 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.046147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42934)
0806 09:03:09.046312 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.046415 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:03:09.046802 22634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.98.61@KRBTEST.COM; defaulting to no policy
I20260806 09:03:09.048400 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.048046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42948)
0806 09:03:09.048190 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.048264 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 09:03:09.048583 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:33867: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
Entry for principal kudu/127.22.98.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab.
I20260806 09:03:09.087395 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:33867
--webserver_interface=127.22.98.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:03:09.097949 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } attempt: 1
I20260806 09:03:09.099247 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.098762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42960)
0806 09:03:09.098987 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.099088 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 09:03:09.100422 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.100009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42976)
0806 09:03:09.100172 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.100246 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:03:09.100688 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:33867: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
I20260806 09:03:09.168988 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } attempt: 2
I20260806 09:03:09.170388 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.169808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42980)
0806 09:03:09.170054 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.170193 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 09:03:09.171502 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.171060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42992)
0806 09:03:09.171221 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.171329 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:03:09.171772 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:33867: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
I20260806 09:03:09.263072 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } attempt: 3
I20260806 09:03:09.264398 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.263876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:42996)
0806 09:03:09.264116 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.264237 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 09:03:09.265614 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.265191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43012)
0806 09:03:09.265370 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.265449 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 09:03:09.265844 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:33867: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
I20260806 09:03:09.409135 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } attempt: 4
I20260806 09:03:09.410422 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.409873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43028)
0806 09:03:09.410190 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.410281 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 09:03:09.411520 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.411148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43040)
0806 09:03:09.411310 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.411383 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 09:03:09.411810 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.61:33867: Network error: Client connection negotiation failed: client connection to 127.22.98.61:33867: connect: Connection refused (error 111)
W20260806 09:03:09.446065 22719 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:09.446460 22719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:09.446563 22719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:09.446688 22719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:09.454314 22719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:09.454437 22719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:09.454519 22719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:09.454588 22719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:09.454684 22719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:09.465159 22719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.61.keytab
--principal=kudu/127.22.98.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.61:33867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.98.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:09.466610 22719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:09.468024 22719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:09.480473 22725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:09.480580 22727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:09.480644 22730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:09.481909 22719 server_base.cc:1034] running on GCE node
I20260806 09:03:09.482741 22719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:09.484346 22719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:09.485585 22719 hybrid_clock.cc:648] HybridClock initialized: now 1786006989485510 us; error 46 us; skew 500 ppm
I20260806 09:03:09.485939 22719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:09 dist-test-slave-jpj1 krb5kdc[22593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:09.493512 22719 init.cc:377] Logged in from keytab as kudu/127.22.98.61@KRBTEST.COM (short username kudu)
I20260806 09:03:09.495239 22719 webserver.cc:460] Webserver started at http://127.22.98.61:44683/ using document root <none> and password file <none>
I20260806 09:03:09.496047 22719 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:09.496145 22719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:09.496484 22719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:09.499876 22719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data/instance:
uuid: "6daa64ea67ec4072accdcb9ecb7af483"
format_stamp: "Formatted at 2026-08-06 09:03:09 on dist-test-slave-jpj1"
server_key: "e5d564c5e215408869af8b7407479fe1"
server_key_iv: "97dd57a8476a25b6dfd4c1c5d6641184"
server_key_version: "encryptionkey@0"
I20260806 09:03:09.500800 22719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal/instance:
uuid: "6daa64ea67ec4072accdcb9ecb7af483"
format_stamp: "Formatted at 2026-08-06 09:03:09 on dist-test-slave-jpj1"
server_key: "e5d564c5e215408869af8b7407479fe1"
server_key_iv: "97dd57a8476a25b6dfd4c1c5d6641184"
server_key_version: "encryptionkey@0"
I20260806 09:03:09.506583 22719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:03:09.510511 22738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:09.512136 22719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:03:09.512332 22719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal
uuid: "6daa64ea67ec4072accdcb9ecb7af483"
format_stamp: "Formatted at 2026-08-06 09:03:09 on dist-test-slave-jpj1"
server_key: "e5d564c5e215408869af8b7407479fe1"
server_key_iv: "97dd57a8476a25b6dfd4c1c5d6641184"
server_key_version: "encryptionkey@0"
I20260806 09:03:09.512554 22719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:09.532267 22719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:09.536559 22719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:09.536841 22719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:09.555313 22719 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.61:33867
I20260806 09:03:09.555333 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.61:33867 every 8 connection(s)
I20260806 09:03:09.556703 22719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/data/info.pb
I20260806 09:03:09.560490 22817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:09.562587 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22719
I20260806 09:03:09.562774 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-1/wal/instance
I20260806 09:03:09.563230 22920 external_mini_cluster.cc:1428] Setting key cfff4eefc83f6aa24385a15e2d6db5cb
I20260806 09:03:09.567219 22817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:09 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:09.588476 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.569962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.22.98.62:37927)
0806 09:03:09.570458 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:09.570463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:09.571510 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 09:03:09.573481 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:09.573487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:09.573569 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:09.573677 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:09.575311 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.576098 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.577678 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.578129 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.578250 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.581766 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:09.581772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:09.581775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:09.581832 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:09.584908 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.585845 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.585852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.585856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.586001 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.586485 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.586489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.586491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.586828 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:09.586988 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:09.587835 (+ 847us) server_negotiation.cc:299] Negotiation successful
0806 09:03:09.588099 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":280,"threads_started":1}
I20260806 09:03:09.589530 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.570544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37927 (local address 127.0.0.1:50700)
0806 09:03:09.571306 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.571345 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:09.573190 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:09.573953 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:09.573961 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:09.574508 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:09.575072 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.575091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.576265 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.576270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.577518 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.577532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.578369 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.578377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.578601 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.579257 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:09.579281 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:09.581528 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:09.585085 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.585096 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.585116 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.585650 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.586156 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.586160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.586163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.586368 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.586976 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:09.586985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:09.587372 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 09:03:09.587688 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 09:03:09.588275 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":314,"threads_started":1}
I20260806 09:03:09.591480 22817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.98.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.98.60@KRBTEST.COM" created.
Aug 06 09:03:09 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:09.605695 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.592173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43052)
0806 09:03:09.592358 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.592379 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:09.592467 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:09.593628 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:09.593632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:09.594081 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:09.594235 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.594245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.595741 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.595752 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.596637 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.596650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.597443 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.597449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.597614 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.598224 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:09.598241 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:09.600273 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:09.603256 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.603261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.603266 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.603673 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.604013 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.604018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.604022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.604154 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.604740 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:09.604751 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:09.604886 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 09:03:09.605014 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 09:03:09.605302 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 09:03:09.605786 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.592471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.22.98.61:33867)
0806 09:03:09.593185 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:09.593191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:09.593215 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:03:09.593287 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:09.593293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:09.593473 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:09.593630 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:09.594394 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.595610 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.596764 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.597309 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.597419 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.600473 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:09.600485 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:09.600499 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:09.600527 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:09.603133 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.603790 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.603799 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.603801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.603912 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.604264 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.604269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.604271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.604609 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:09.604794 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:09.605183 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 09:03:09.605414 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":421,"threads_started":1}
I20260806 09:03:09.608371 22817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:09.609740 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.609211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60626)
0806 09:03:09.609397 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.609530 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 09:03:09.610016 22747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.611595 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.611233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60630)
0806 09:03:09.611386 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.611463 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 09:03:09.611793 22817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
Entry for principal kudu/127.22.98.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab.
I20260806 09:03:09.639427 22920 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6BBFDc/build/asan/bin/kudu
/tmp/dist-test-task6BBFDc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:42061
--webserver_interface=127.22.98.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.98.20:35469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:03:09.661100 22817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 1
I20260806 09:03:09.662640 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.661976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60640)
0806 09:03:09.662213 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.662368 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 09:03:09.663906 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.663473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60654)
0806 09:03:09.663653 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.663747 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 09:03:09.664232 22817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.711078 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } attempt: 5
Aug 06 09:03:09 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:09.725080 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.711825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43060)
0806 09:03:09.712057 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.712081 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:09.712219 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:09.712599 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:09.712604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:09.713139 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:09.713389 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.713404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.714440 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.714446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.715882 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:09.715909 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.716742 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.716748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:09.716971 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.717657 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:09.717676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:09.719685 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:09.722507 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.722513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.722517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.722894 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.723244 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:09.723249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:09.723252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:09.723390 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:09.724105 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:09.724110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:09.724250 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 09:03:09.724404 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 09:03:09.724692 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 09:03:09.725088 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.711982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.22.98.61:33867)
0806 09:03:09.712158 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:09.712164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:09.712184 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:03:09.712362 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:09.712366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:09.712451 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:09.712528 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:09.713555 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.714284 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.716066 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:09.716591 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:09.716688 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:09.719872 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:09.719877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:09.719880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:09.719913 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:09.722387 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.723006 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.723012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.723016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.723140 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:09.723496 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:09.723501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:09.723505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:09.723979 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:09.724114 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:09.724549 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 09:03:09.724780 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 09:03:09.726106 22709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:09.727174 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.726780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60666)
0806 09:03:09.726948 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.727027 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:03:09.728135 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.727817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60680)
0806 09:03:09.727956 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.728025 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 09:03:09.728374 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.732456 22817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 2
I20260806 09:03:09.733422 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.732971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60690)
0806 09:03:09.733152 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.733253 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 09:03:09.734465 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.734081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60692)
0806 09:03:09.734239 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.734326 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 09:03:09.734715 22817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.779645 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 1
I20260806 09:03:09.780805 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.780314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60704)
0806 09:03:09.780510 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.780620 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 09:03:09.781937 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.781586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60710)
0806 09:03:09.781744 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.781821 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:03:09.782186 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.825990 22817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 3
I20260806 09:03:09.827450 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.826800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60724)
0806 09:03:09.827081 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.827219 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 09:03:09.828747 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.828230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60734)
0806 09:03:09.828493 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.828596 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260806 09:03:09.829020 22817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.850404 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 2
I20260806 09:03:09.851728 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.851185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60746)
0806 09:03:09.851410 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.851515 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 09:03:09.852836 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.852417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60760)
0806 09:03:09.852606 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.852697 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 09:03:09.853067 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.959299 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 3
I20260806 09:03:09.960620 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.960096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60764)
0806 09:03:09.960357 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.960451 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 09:03:09.961746 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.961339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60776)
0806 09:03:09.961553 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.961627 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 09:03:09.961958 22709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
I20260806 09:03:09.972223 22817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 4
I20260806 09:03:09.973402 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.972817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60790)
0806 09:03:09.973082 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.973210 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 09:03:09.974476 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:09.974071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60792)
0806 09:03:09.974239 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:09.974332 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 09:03:09.974757 22817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.98.60:42061: Network error: Client connection negotiation failed: client connection to 127.22.98.60:42061: connect: Connection refused (error 111)
W20260806 09:03:09.998988 22830 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:03:09.999339 22830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:03:09.999437 22830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:03:09.999580 22830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:03:10.006916 22830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:03:10.007027 22830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:03:10.007123 22830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:03:10.007205 22830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:03:10.007320 22830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:03:10.017552 22830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.98.20:35469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.98.62:37927,127.22.98.61:33867,127.22.98.60:42061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/krb5kdc/kudu_127.22.98.60.keytab
--principal=kudu/127.22.98.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.98.60:42061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.98.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 08:04:30 UTC on ccycloud.quasar-tkivct.root.comops.site
build id 4569871
ASAN enabled
I20260806 09:03:10.018872 22830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:03:10.020242 22830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:03:10.031910 22839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:10.031953 22841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:03:10.031874 22838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:03:10.032996 22830 server_base.cc:1034] running on GCE node
I20260806 09:03:10.033739 22830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:03:10.035120 22830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:03:10.036377 22830 hybrid_clock.cc:648] HybridClock initialized: now 1786006990036307 us; error 37 us; skew 500 ppm
I20260806 09:03:10.036728 22830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:03:10.039978 22830 init.cc:377] Logged in from keytab as kudu/127.22.98.60@KRBTEST.COM (short username kudu)
I20260806 09:03:10.041528 22830 webserver.cc:460] Webserver started at http://127.22.98.60:42581/ using document root <none> and password file <none>
I20260806 09:03:10.042222 22830 fs_manager.cc:359] Metadata directory not provided
I20260806 09:03:10.042325 22830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:03:10.042635 22830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:03:10.045604 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data/instance:
uuid: "3b328cac4d5447ea9213179806760341"
format_stamp: "Formatted at 2026-08-06 09:03:10 on dist-test-slave-jpj1"
server_key: "2871e171c030bbf0ea1886abd653d8d4"
server_key_iv: "c5b8186a8446ea6ae59da3210b43023b"
server_key_version: "encryptionkey@0"
I20260806 09:03:10.046387 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal/instance:
uuid: "3b328cac4d5447ea9213179806760341"
format_stamp: "Formatted at 2026-08-06 09:03:10 on dist-test-slave-jpj1"
server_key: "2871e171c030bbf0ea1886abd653d8d4"
server_key_iv: "c5b8186a8446ea6ae59da3210b43023b"
server_key_version: "encryptionkey@0"
I20260806 09:03:10.051644 22830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:03:10.055176 22853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:03:10.056587 22830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:03:10.056797 22830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data,/tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal
uuid: "3b328cac4d5447ea9213179806760341"
format_stamp: "Formatted at 2026-08-06 09:03:10 on dist-test-slave-jpj1"
server_key: "2871e171c030bbf0ea1886abd653d8d4"
server_key_iv: "c5b8186a8446ea6ae59da3210b43023b"
server_key_version: "encryptionkey@0"
I20260806 09:03:10.057003 22830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:03:10.068727 22830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:03:10.072829 22830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:03:10.073083 22830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:03:10.090762 22830 rpc_server.cc:307] RPC server started. Bound to: 127.22.98.60:42061
I20260806 09:03:10.090793 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.98.60:42061 every 8 connection(s)
I20260806 09:03:10.092235 22830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/data/info.pb
I20260806 09:03:10.096259 22940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:03:10.099028 22920 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6BBFDc/build/asan/bin/kudu as pid 22830
I20260806 09:03:10.099212 22920 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6BBFDc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786006610327111-22920-0/minicluster-data/master-2/wal/instance
I20260806 09:03:10.099687 22920 external_mini_cluster.cc:1428] Setting key 025bcb5bea1a91dac032ac81fc79f2fe
I20260806 09:03:10.103946 22940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:10.121891 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.106709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.22.98.62:37927)
0806 09:03:10.107247 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.107253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.107622 (+ 369us) server_negotiation.cc:407] Connection header received
0806 09:03:10.107819 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.107825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.107924 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.108072 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.109305 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.110048 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.111450 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.111886 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.111997 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.115516 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.115523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.115528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.115602 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.119031 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.119784 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.119788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.119791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.119919 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.120303 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.120307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.120309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.120566 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.120711 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.121264 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.121485 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
I20260806 09:03:10.122879 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.101491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.22.98.60:42061)
0806 09:03:10.102284 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.102297 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.102343 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:03:10.104113 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.104140 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.104626 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.105022 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.107212 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.108930 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.110247 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.110940 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.111117 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.115271 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.115280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.115294 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.115328 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.119143 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.119826 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.119835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.119839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.119993 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.120371 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.120376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.120380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.120899 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.121202 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.121518 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.121843 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":572,"thread_start_us":350,"threads_started":1}
I20260806 09:03:10.122916 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.106700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.62:37927 (local address 127.0.0.1:50706)
0806 09:03:10.107453 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:10.107500 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:10.107695 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:10.108170 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:10.108175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:10.108852 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:10.109137 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.109147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.110215 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.110219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.111297 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.111310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.112112 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.112121 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.112254 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.112821 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:10.112848 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:10.115290 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:10.119173 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.119184 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.119194 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.119660 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.120025 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.120030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.120033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.120193 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.120731 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:10.120740 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:10.120934 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 09:03:10.121159 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 09:03:10.121690 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":321,"threads_started":1}
I20260806 09:03:10.124586 22940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.62@KRBTEST.COM
I20260806 09:03:10.138228 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.125209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.61:33867 (local address 127.0.0.1:43062)
0806 09:03:10.125378 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:10.125397 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:10.125474 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:10.125958 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:10.125963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:10.126365 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:10.126556 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.126568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.127523 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.127529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.128502 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.128515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.129225 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.129232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.129490 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.130144 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:10.130168 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:10.132347 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:10.135912 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.135916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.135919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.136283 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.136689 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.136692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.136694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.136853 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.137461 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:10.137465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:10.137601 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 09:03:10.137710 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 09:03:10.137918 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 09:03:10.138538 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.125455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.22.98.61:33867)
0806 09:03:10.125636 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.125643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.125660 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:03:10.125718 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.125722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.125798 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.125880 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.126699 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.127381 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.128662 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.129077 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.129187 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.132531 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.132537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.132540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.132581 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.135769 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.136399 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.136404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.136408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.136573 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.136961 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.136966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.136969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.137327 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.137494 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.137833 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.138044 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 09:03:10.139156 22709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 4
I20260806 09:03:10.139678 22940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:03:10.141867 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.126296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.22.98.62:37927)
0806 09:03:10.126466 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.126472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.126490 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:03:10.126651 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.126657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.126742 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.126825 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.127993 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.128660 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.129908 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.130524 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.130717 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.134959 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.134965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.134970 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.135009 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.137980 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.138792 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.138797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.138801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.138929 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.139373 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.139378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.139381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.139809 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.140094 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.141336 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.141543 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.62@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006990, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.60@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.98.61@KRBTEST.COM
I20260806 09:03:10.155247 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.139815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60810)
0806 09:03:10.140076 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:10.140096 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:10.140195 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:10.140694 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:10.140699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:10.141194 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:10.141443 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.141459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.142659 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.142665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.144093 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.144110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.144944 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.144948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.145095 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.145614 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:10.145631 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:10.147996 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:10.152401 (+ 4405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.152406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.152410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.152900 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.153345 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.153350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.153355 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.153552 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.154221 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:10.154226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:10.154392 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 09:03:10.154549 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 09:03:10.154853 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260806 09:03:10.155247 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.139973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.22.98.60:42061)
0806 09:03:10.140154 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.140159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.140176 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:03:10.140353 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.140358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.140425 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.140515 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.141602 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.142495 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.144236 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.144774 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.144922 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.148167 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.148172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.148176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.148204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.152240 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.153037 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.153041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.153043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.153198 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.153718 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.153723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.153726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.154091 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.154241 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.154663 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.154902 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":64}
I20260806 09:03:10.157317 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.140386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60812)
0806 09:03:10.140540 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:10.140557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:10.140645 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:10.142701 (+ 2056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:10.142705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:10.143193 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:10.143379 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.143391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.144756 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.144761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.146171 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.146186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.147037 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.147042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.147240 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.148830 (+ 1590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:10.148855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:10.151162 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:10.154603 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.154607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.154611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.155063 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.155592 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.155597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.155600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.155760 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.156453 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:10.156458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:10.156680 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 09:03:10.156852 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 09:03:10.157087 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 09:03:10.158176 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.140641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60812 (local address 127.22.98.60:42061)
0806 09:03:10.142438 (+ 1797us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.142444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.142471 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:03:10.142515 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.142519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.142582 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.142682 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.143590 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.144618 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.146334 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.146905 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.147014 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.151361 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.151366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.151369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.151396 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.154487 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.155216 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.155220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.155224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.155448 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.155888 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.155892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.155895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.156274 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.156406 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.157610 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.157813 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1670,"thread_start_us":1098,"threads_started":1}
I20260806 09:03:10.160802 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.143648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.22.98.61:33867)
0806 09:03:10.143805 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.143811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.143922 (+ 111us) server_negotiation.cc:407] Connection header received
0806 09:03:10.144066 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.144071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.144134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.144222 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.146518 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.147518 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.148705 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.149322 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.149460 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.154180 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.154186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.154190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.154237 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.157296 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.158181 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.158187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.158191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.158623 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.159068 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.159073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.159077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.159423 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.159692 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.160282 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.160496 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:03:10.161758 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: Bootstrap starting.
I20260806 09:03:10.162621 22940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: Bootstrap starting.
I20260806 09:03:10.165660 22709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:10.165853 22940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:10.166692 22920 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:03:10.166918 22709 log.cc:824] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:10.167013 22940 log.cc:824] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:10.170693 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: No bootstrap required, opened a new log
I20260806 09:03:10.170691 22940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: No bootstrap required, opened a new log
I20260806 09:03:10.169660 22674 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4eeae5d834d447d68f4c5c5be9946be4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50718
I20260806 09:03:10.178748 22709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.179175 22709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eeae5d834d447d68f4c5c5be9946be4, State: Initialized, Role: FOLLOWER
I20260806 09:03:10.179771 22709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.180819 22954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.181216 22954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.181207 22940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.181644 22940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b328cac4d5447ea9213179806760341, State: Initialized, Role: FOLLOWER
I20260806 09:03:10.182224 22709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:10.182264 22674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 09:03:10.182148 22940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.182456 22674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:10.183076 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.183602 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.184383 22940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:10.185554 22674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.187884 22896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eeae5d834d447d68f4c5c5be9946be4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b328cac4d5447ea9213179806760341"
I20260806 09:03:10.188349 22896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:10.191794 22896 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eeae5d834d447d68f4c5c5be9946be4 in term 1.
I20260806 09:03:10.192490 22637 leader_election.cc:302] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b328cac4d5447ea9213179806760341, 4eeae5d834d447d68f4c5c5be9946be4; no voters:
I20260806 09:03:10.192968 22954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:03:10.193051 22674 leader_election.cc:288] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [CANDIDATE]: Term 1 election: Requested vote from peers 6daa64ea67ec4072accdcb9ecb7af483 (127.22.98.61:33867), 3b328cac4d5447ea9213179806760341 (127.22.98.60:42061)
I20260806 09:03:10.203197 22954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [term 1 LEADER]: Becoming Leader. State: Replica: 4eeae5d834d447d68f4c5c5be9946be4, State: Running, Role: LEADER
I20260806 09:03:10.203994 22954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.212366 22971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eeae5d834d447d68f4c5c5be9946be4. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.212982 22971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:10.215899 22980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:03:10.230744 22980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 09:03:10.238099 22640 tablet.cc:2369] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:03:10.245187 22995 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 09:03:10.252691 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.217620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.22.98.62:37927)
0806 09:03:10.217822 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.217829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.226158 (+ 8329us) server_negotiation.cc:407] Connection header received
0806 09:03:10.228413 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.228420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.228512 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.228591 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.232028 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.233154 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.235357 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.236059 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.236158 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.239961 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.239968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.239973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.240022 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.243817 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.244670 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.244676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.244680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.244882 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.245418 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.245423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.245426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.245817 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.246099 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.252003 (+ 5904us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.252285 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:03:10.250497 22613 master_runner.cc:186] Error getting master registration for 127.22.98.61:33867: OK, instance_id { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" instance_seqno: 1786006989550236 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260806 09:03:10.266386 23002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:03:10.266539 23002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:03:10.266685 22613 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:03:10.254809 22830 master_runner.cc:186] Error getting master registration for 127.22.98.61:33867: OK, instance_id { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" instance_seqno: 1786006989550236 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:03:10.270231 22830 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:03:10.273957 22817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } attempt: 5
I20260806 09:03:10.282040 22896 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [term 1 FOLLOWER]: Refusing update from remote peer 4eeae5d834d447d68f4c5c5be9946be4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:03:10.283195 22971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 06 09:03:10 dist-test-slave-jpj1 krb5kdc[22593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786006989, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.98.61@KRBTEST.COM for kudu/127.22.98.60@KRBTEST.COM
I20260806 09:03:10.292871 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.224337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.22.98.60:42061)
0806 09:03:10.224584 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.224592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.231511 (+ 6919us) server_negotiation.cc:407] Connection header received
0806 09:03:10.235978 (+ 4467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.235985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.236099 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.236170 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.242409 (+ 6239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.243592 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.255382 (+ 11790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.256170 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.256339 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.272699 (+ 16360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.272709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.272713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.272759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.276283 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.277706 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.277711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.277715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.277882 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.281235 (+ 3353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.281241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.281244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.281660 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.282427 (+ 767us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.292223 (+ 9796us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.292483 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:03:10.297465 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.274616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.22.98.60:42061)
0806 09:03:10.275186 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.275191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.275213 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:03:10.275268 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.275273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.275354 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.275461 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.276606 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.277728 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.279613 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.280434 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.280567 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.286567 (+ 6000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.286575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.286580 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.286619 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.290936 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.292330 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.292336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.292368 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.292665 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.293438 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.293443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.293447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.294233 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.294597 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.295431 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.296144 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445}
I20260806 09:03:10.300690 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.274538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.98.60:42061 (local address 127.0.0.1:60844)
0806 09:03:10.274851 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 09:03:10.274870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:03:10.274975 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:03:10.275696 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:03:10.275701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:03:10.276199 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:03:10.276422 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.276437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.277929 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.277935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.279395 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:03:10.279414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.281224 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.281231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:03:10.281445 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.282256 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:03:10.282300 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:03:10.286269 (+ 3969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:03:10.291207 (+ 4938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.291212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.291216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.292112 (+ 896us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.292930 (+ 818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:03:10.292934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:03:10.292938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:03:10.293243 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:03:10.294522 (+ 1279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:03:10.294528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:03:10.294913 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 09:03:10.295714 (+ 801us) client_negotiation.cc:240] Negotiation successful
0806 09:03:10.296761 (+ 1047us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 09:03:10.304844 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:10.226755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.22.98.61:33867)
0806 09:03:10.226949 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.226955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.232129 (+ 5174us) server_negotiation.cc:407] Connection header received
0806 09:03:10.236413 (+ 4284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.236419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.236507 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.236582 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.243813 (+ 7231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.244881 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.264379 (+ 19498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:10.265095 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:10.265262 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:10.269357 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:03:10.269365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:03:10.269370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:03:10.269430 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:03:10.273370 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.281768 (+ 8398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.281775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.281778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.281972 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:03:10.301860 (+ 19888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:03:10.301868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:03:10.301872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:03:10.302295 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:03:10.302565 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:10.303714 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0806 09:03:10.304011 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:03:10.309064 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: Bootstrap starting.
I20260806 09:03:10.313196 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eeae5d834d447d68f4c5c5be9946be4. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.313591 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.324985 22817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: Neither blocks nor log segments found. Creating new log.
I20260806 09:03:10.329241 22971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.329564 22971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:10.328985 22817 log.cc:824] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: Log is configured to *not* fsync() on all Append() calls
I20260806 09:03:10.336109 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.336419 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.339488 22817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: No bootstrap required, opened a new log
I20260806 09:03:10.352068 22980 catalog_manager.cc:1347] Generated new cluster ID: e2acd1337bf04089842af0c9c4f76bd9
I20260806 09:03:10.352226 22980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:03:10.353427 22817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.353927 22817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6daa64ea67ec4072accdcb9ecb7af483, State: Initialized, Role: FOLLOWER
I20260806 09:03:10.354493 22817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } }
I20260806 09:03:10.355855 23014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.356521 23014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.357116 22817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:03:10.357502 22784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:03:10.357796 22786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4eeae5d834d447d68f4c5c5be9946be4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6daa64ea67ec4072accdcb9ecb7af483"
I20260806 09:03:10.361629 22784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [term 1 FOLLOWER]: Refusing update from remote peer 4eeae5d834d447d68f4c5c5be9946be4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:03:10.365461 22786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eeae5d834d447d68f4c5c5be9946be4 in term 1.
I20260806 09:03:10.368299 22954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:03:10.387487 22980 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:03:10.389546 22980 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:03:10.416217 23014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eeae5d834d447d68f4c5c5be9946be4. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.416591 23014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.416711 22971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.417016 22971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:03:10.417775 23018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eeae5d834d447d68f4c5c5be9946be4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eeae5d834d447d68f4c5c5be9946be4" member_type: VOTER last_known_addr { host: "127.22.98.62" port: 37927 } } peers { permanent_uuid: "6daa64ea67ec4072accdcb9ecb7af483" member_type: VOTER last_known_addr { host: "127.22.98.61" port: 33867 } } peers { permanent_uuid: "3b328cac4d5447ea9213179806760341" member_type: VOTER last_known_addr { host: "127.22.98.60" port: 42061 } } }
I20260806 09:03:10.418201 23018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:03:10.422328 22980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eeae5d834d447d68f4c5c5be9946be4: Generated new TSK 0
I20260806 09:03:10.423491 22980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:03:10.487525 23036 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 09:03:10.529168 22947 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.519463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.22.98.62:37927)
0806 09:03:10.519708 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.519715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.526775 (+ 7060us) server_negotiation.cc:407] Connection header received
0806 09:03:10.527347 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.527354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.527464 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.527586 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.528890 (+ 1304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50724: BlockingRecv error: recv got EOF from 127.0.0.1:50724 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 09:03:10.531463 22945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.519241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.22.98.60:42061)
0806 09:03:10.519438 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.519445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.529798 (+ 10353us) server_negotiation.cc:407] Connection header received
0806 09:03:10.530038 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.530045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.530159 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.530238 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.531212 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60852: BlockingRecv error: recv got EOF from 127.0.0.1:60852 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 09:03:10.533955 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.531824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.22.98.61:33867)
0806 09:03:10.532017 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.532023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.532615 (+ 592us) server_negotiation.cc:407] Connection header received
0806 09:03:10.532737 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.532744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.532836 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.532942 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.533682 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43076: BlockingRecv error: recv got EOF from 127.0.0.1:43076 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 09:03:10.545773 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.537943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.22.98.61:33867)
0806 09:03:10.538104 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.538110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.544098 (+ 5988us) server_negotiation.cc:407] Connection header received
0806 09:03:10.544365 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.544371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.544500 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.544602 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.545475 (+ 873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43092: BlockingRecv error: recv got EOF from 127.0.0.1:43092 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 09:03:10.562377 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.553900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.22.98.61:33867)
0806 09:03:10.554102 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.554108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.560442 (+ 6334us) server_negotiation.cc:407] Connection header received
0806 09:03:10.560808 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.560815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.560933 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.561012 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.562158 (+ 1146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43098: BlockingRecv error: recv got EOF from 127.0.0.1:43098 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 09:03:10.590101 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.587509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.22.98.61:33867)
0806 09:03:10.587755 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.587762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.588372 (+ 610us) server_negotiation.cc:407] Connection header received
0806 09:03:10.588592 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.588597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.588714 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.588844 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.589841 (+ 997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43110: BlockingRecv error: recv got EOF from 127.0.0.1:43110 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260806 09:03:10.599838 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.597475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.22.98.61:33867)
0806 09:03:10.597690 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.597696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.598262 (+ 566us) server_negotiation.cc:407] Connection header received
0806 09:03:10.598468 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.598474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.598591 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.598699 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.599593 (+ 894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43118: BlockingRecv error: recv got EOF from 127.0.0.1:43118 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260806 09:03:10.613519 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.611302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.22.98.61:33867)
0806 09:03:10.611498 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.611504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.612065 (+ 561us) server_negotiation.cc:407] Connection header received
0806 09:03:10.612283 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.612289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.612384 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.612493 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.613289 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43128: BlockingRecv error: recv got EOF from 127.0.0.1:43128 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 09:03:10.636051 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.633559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.22.98.61:33867)
0806 09:03:10.633782 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.633789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.634381 (+ 592us) server_negotiation.cc:407] Connection header received
0806 09:03:10.634608 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.634614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.634721 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.634850 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.635798 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43144: BlockingRecv error: recv got EOF from 127.0.0.1:43144 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 09:03:10.675320 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.672773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.22.98.61:33867)
0806 09:03:10.672969 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.672976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.673558 (+ 582us) server_negotiation.cc:407] Connection header received
0806 09:03:10.673806 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.673813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.673917 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.674035 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.675035 (+ 1000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43148: BlockingRecv error: recv got EOF from 127.0.0.1:43148 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 09:03:10.745273 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.743297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.22.98.61:33867)
0806 09:03:10.743498 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.743506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.743908 (+ 402us) server_negotiation.cc:407] Connection header received
0806 09:03:10.744105 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.744110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.744187 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.744283 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.745052 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47194: BlockingRecv error: recv got EOF from 127.0.0.1:47194 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 09:03:10.878727 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:10.876964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.22.98.61:33867)
0806 09:03:10.877235 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:10.877243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:10.877365 (+ 122us) server_negotiation.cc:407] Connection header received
0806 09:03:10.877582 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:10.877588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:10.877673 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:10.877766 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:10.878519 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47200: BlockingRecv error: recv got EOF from 127.0.0.1:47200 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260806 09:03:11.140611 22827 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:03:11.138749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.22.98.61:33867)
0806 09:03:11.139009 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.139015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.139138 (+ 123us) server_negotiation.cc:407] Connection header received
0806 09:03:11.139375 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.139380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.139455 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.139574 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:03:11.140405 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47214: BlockingRecv error: recv got EOF from 127.0.0.1:47214 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 09:03:11.268687 23002 catalog_manager.cc:1259] Loaded cluster ID: e2acd1337bf04089842af0c9c4f76bd9
I20260806 09:03:11.268817 23002 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: loading cluster ID for follower catalog manager: success
I20260806 09:03:11.271464 23002 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: acquiring CA information for follower catalog manager: success
I20260806 09:03:11.273264 23002 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b328cac4d5447ea9213179806760341: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:03:11.492308 23036 catalog_manager.cc:1259] Loaded cluster ID: e2acd1337bf04089842af0c9c4f76bd9
I20260806 09:03:11.492430 23036 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: loading cluster ID for follower catalog manager: success
I20260806 09:03:11.495064 23036 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: acquiring CA information for follower catalog manager: success
I20260806 09:03:11.496919 23036 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6daa64ea67ec4072accdcb9ecb7af483: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:03:11.665934 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:11.658505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.22.98.61:33867)
0806 09:03:11.659304 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.659309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.659328 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:03:11.659391 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.659395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.659576 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.659700 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:11.660260 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.661271 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.663299 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.663864 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.663974 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:11.664597 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:11.665266 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:11.665386 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:11.665521 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 09:03:11.665598 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":532,"threads_started":1}
I20260806 09:03:11.678386 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:11.672218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.22.98.60:42061)
0806 09:03:11.673149 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.673153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.673169 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:03:11.673275 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.673279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.673431 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.673579 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:11.674020 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.674758 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.676120 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.676540 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.676680 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:11.677225 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:11.677744 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:11.677880 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:11.677996 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 09:03:11.678084 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":722,"threads_started":1}
I20260806 09:03:11.707782 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:11.685817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.22.98.62:37927)
0806 09:03:11.698887 (+ 13070us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.698893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.698915 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:03:11.699046 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.699052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.699268 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.699461 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:11.700426 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.701545 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.704640 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.705286 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.705424 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:11.706219 (+ 795us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:11.706693 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:11.706821 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:11.707323 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 09:03:11.707420 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12932,"thread_start_us":13795,"threads_started":1}
I20260806 09:03:11.709074 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:11.686002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.22.98.60:42061)
0806 09:03:11.686197 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.686205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.701664 (+ 15459us) server_negotiation.cc:407] Connection header received
0806 09:03:11.701829 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.701836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.701952 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.702040 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:11.702944 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.704133 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.706283 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.706902 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.706995 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:11.708204 (+ 1209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:11.708343 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:11.708409 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:11.708712 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 09:03:11.708802 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:03:11.712211 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:03:11.698572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.22.98.61:33867)
0806 09:03:11.698743 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 09:03:11.698750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:03:11.704373 (+ 5623us) server_negotiation.cc:407] Connection header received
0806 09:03:11.704561 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:03:11.704567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:03:11.704740 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:03:11.704835 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:03:11.705638 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.706828 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.709193 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:03:11.709808 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:03:11.709886 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:03:11.711383 (+ 1497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:03:11.711515 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:03:11.711664 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 09:03:11.711815 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 09:03:11.711907 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 09:03:11.715618 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22613
I20260806 09:03:11.716917 22702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:11.861770 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22613
I20260806 09:03:11.889426 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22719
I20260806 09:03:11.890609 22812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:11.996902 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22719
I20260806 09:03:12.023311 22920 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6BBFDc/build/asan/bin/kudu pid 22830
I20260806 09:03:12.024577 22932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:03:12.132596 22920 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6BBFDc/build/asan/bin/kudu with pid 22830
2026-08-06T09:03:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5134 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9832 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (381711 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST